Skip to content

Improve performance of GetConfigurationChanges (parallel fetch + O(N+M) lookup)#23

Open
ruodanxie wants to merge 1 commit into
mainfrom
user/ruodanxie/improve_get_configuration_changes
Open

Improve performance of GetConfigurationChanges (parallel fetch + O(N+M) lookup)#23
ruodanxie wants to merge 1 commit into
mainfrom
user/ruodanxie/improve_get_configuration_changes

Conversation

@ruodanxie
Copy link
Copy Markdown

Summary

Improves the performance of AppConfigurationImporter.GetConfigurationChanges through two internal changes. Public API surface, return value shape, and observable behavior are unchanged for all existing source types and scenarios.

Motivation

Currently, for example, when importing from configuration file which has 2000 KVs and target store has 3 KVs, total time taken is 1.36s nearly. However, when importing from configuration file which has 2000 KVs and target store also has 2000 KVs, it took 12s approximately. There is room for optimization.

The following are the reasons why running GetConfigurationChanges is currently slow:

  1. Source-side fetch and target-side listing ran sequentially, so their latencies added up.
  2. The internal diff used Array.find + Array.splice on the source array for every target item, producing O(N × M) complexity that degrades quickly when both stores grow.

Changes

All changes are internal to GetConfigurationChanges in libraries/azure-app-configuration-importer/src/appConfigurationImporter.ts.

1. Parallel source and target fetch

  • sourcePromise (source GetConfigurationSettings()) and targetPromise (target store listConfigurationSettings) are now kicked off at the same time and awaited later.

2. O(N + M) diff using Map and Set

  • Replaced the nested-object lookup (srcKeyLabelLookUp[key][label]) with srcMap: Map<"key\u0000label", incoming>.
  • Replaced configurationSettingToAdd (array + Array.find + splice(indexOf(...))) with toAddKeys: Set<"key\u0000label">.
  • The composite key uses the NUL character (\u0000) as the separator since it cannot appear in App Configuration keys or labels, guaranteeing uniqueness.

Version bump

Bumps libraries/azure-app-configuration-importer from 3.0.0-preview to 3.0.0-preview.2 (next preview iteration; no public API additions).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant