Setup CosmosDB Linux Docker container to run tests on CI #6683

bouwkast · 2025-02-18T22:32:47Z

Summary of changes

This adds a cosmosdb-emulator to the docker-compose.yml so that we can run the tests on CI.

Reason for change

For each update to the CosmosDB NuGet I manually test it out to see if our instrumentation still works, I'd like to not have to do that.

Implementation details

Followed: https://learn.microsoft.com/en-us/azure/cosmos-db/how-to-develop-emulator?pivots=api-nosql&tabs=docker-linux%2Ccsharp

and

https://learn.microsoft.com/en-us/azure/cosmos-db/emulator

Test coverage

Now there are tests in CI again (hopefully)
Ran locally with the latest version and seemed good

Other details

cosmosdb-emulator  | This is an evaluation version.  There are [88] days left in the evaluation period.
cosmosdb-emulator  | 2.14.21.0 (1c783d8f)
cosmosdb-emulator  | Copyright (C) Microsoft Corporation. All rights reserved.

We only have 88 days until I need to run them manually again 😭

The solution to this is to apparently delete the container image and pull it back down.

docker compose down
docker rmi mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
docker compose up cosmosdb-emulator

Unsure if the days will actually go down though 😅

bouwkast · 2025-02-18T22:43:24Z

docker-compose.yml

+    image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
+    container_name: cosmosdb-emulator
+    environment:
+      - AZURE_COSMOS_EMULATOR_PARTITION_COUNT=2


I think the default is 10

But we only have LastName as a partition key, so decreasing seems fine IMO

bouwkast · 2025-02-18T22:44:05Z

docker-compose.yml

I'm unsure if an addition like this requires a VM update?

Yeah, generally we do, as it avoids a whole category of flake (network issue, API limits). It's not critical to do ahead of time though, and if we do the service bus one too, then we can update them at that point

bouwkast · 2025-02-18T22:45:41Z

tracer/test/Datadog.Trace.ClrProfiler.IntegrationTests/CosmosTests.cs

@@ -34,13 +33,12 @@ public static IEnumerable<object[]> GetEnabledConfig()

        public override Result ValidateIntegrationSpan(MockSpan span, string metadataSchemaVersion) => span.IsCosmosDb(metadataSchemaVersion);

-        [SkippableTheory]
+        [Theory]
        [MemberData(nameof(GetEnabledConfig))]
        [Trait("Category", "EndToEnd")]
        [Trait("RunOnWindows", "True")]


So I think the tests are still meant to run only on Windows, but the linux docker container means it could run on all platforms?

I wasn't sure exactly what we want to do here

Yeah, this gives a big flip. Currently the tests assume we're using the powershell-started cosmos emulator, so can only run on Windows. With this change, we can only run on Linux, so we need to remove the RunOnWindows config for now.

Also, you haven't configured the docker-compose for arm64 yet (there's separate testing and start-dependencies stages), so we'll need to do that.

Suggested change

[Trait("RunOnWindows", "True")]

bouwkast · 2025-02-18T22:58:51Z

Ahh the older versions of the NuGet don't contain the HttpClientFactory option

andrewlock · 2025-02-18T23:08:06Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6683) - mean (69ms)  : 66, 73
     .   : milestone, 69,
    master - mean (69ms)  : 65, 73
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6683) - mean (1,002ms)  : 977, 1027
     .   : milestone, 1002,
    master - mean (1,001ms)  : 978, 1024
     .   : milestone, 1001,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6683) - mean (102ms)  : 100, 105
     .   : milestone, 102,
    master - mean (102ms)  : 100, 105
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (6683) - mean (680ms)  : 663, 697
     .   : milestone, 680,
    master - mean (676ms)  : 659, 693
     .   : milestone, 676,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6683) - mean (90ms)  : 88, 92
     .   : milestone, 90,
    master - mean (89ms)  : 87, 91
     .   : milestone, 89,

    section CallTarget+Inlining+NGEN
    This PR (6683) - mean (638ms)  : 620, 656
     .   : milestone, 638,
    master - mean (632ms)  : 616, 648
     .   : milestone, 632,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6683) - mean (191ms)  : 188, 195
     .   : milestone, 191,
    master - mean (191ms)  : 187, 195
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6683) - mean (1,110ms)  : 1084, 1136
     .   : milestone, 1110,
    master - mean (1,108ms)  : 1072, 1144
     .   : milestone, 1108,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6683) - mean (271ms)  : 267, 275
     .   : milestone, 271,
    master - mean (270ms)  : 264, 276
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (6683) - mean (864ms)  : 838, 889
     .   : milestone, 864,
    master - mean (863ms)  : 834, 892
     .   : milestone, 863,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6683) - mean (263ms)  : 260, 267
     .   : milestone, 263,
    master - mean (263ms)  : 259, 267
     .   : milestone, 263,

    section CallTarget+Inlining+NGEN
    This PR (6683) - mean (849ms)  : 808, 890
     .   : milestone, 849,
    master - mean (844ms)  : 806, 882
     .   : milestone, 844,

andrewlock · 2025-02-18T23:29:52Z

Benchmarks Report for tracer 🐌

Benchmarks for #6683 compared to master:

3 benchmarks are faster, with geometric mean 1.136
1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.86μs	43ns	262ns	0.0203	0.00811	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.1μs	55.5ns	324ns	0.0199	0.00996	0	5.8 KB
master	`StartStopWithChild`	net472	15.9μs	38.1ns	142ns	1.04	0.307	0.0944	6.21 KB
#6683	`StartStopWithChild`	net6.0	7.84μs	41.8ns	229ns	0.0154	0.00769	0	5.61 KB
#6683	`StartStopWithChild`	netcoreapp3.1	9.94μs	52.8ns	274ns	0.0287	0.0143	0	5.81 KB
#6683	`StartStopWithChild`	net472	16.4μs	41.7ns	161ns	1.05	0.312	0.0961	6.2 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	492μs	486ns	1.82μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	700μs	282ns	1.05μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	848μs	626ns	2.34μs	0.422	3.3 KB
#6683	`WriteAndFlushEnrichedTraces`	net6.0	486μs	453ns	1.7μs	0	2.7 KB
#6683	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	707μs	545ns	2.11μs	0	2.7 KB
#6683	`WriteAndFlushEnrichedTraces`	net472	857μs	1.57μs	6.09μs	0.425	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	132μs	417ns	1.61μs	0.135	14.47 KB
master	`SendRequest`	netcoreapp3.1	147μs	396ns	1.53μs	0.22	17.27 KB
master	`SendRequest`	net472	0.00199ns	0.000288ns	0.00112ns	0	0 b
#6683	`SendRequest`	net6.0	132μs	559ns	2.17μs	0.128	14.47 KB
#6683	`SendRequest`	netcoreapp3.1	149μs	210ns	815ns	0.222	17.27 KB
#6683	`SendRequest`	net472	0.000765ns	0.000326ns	0.00122ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #6683

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.87 KB	41.49 KB	-381 B	-0.91%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	608μs	3.32μs	18.8μs	0.604	0	0	41.87 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	670μs	3.78μs	27.5μs	0.338	0	0	41.69 KB
master	`WriteAndFlushEnrichedTraces`	net472	840μs	2.83μs	10.6μs	8.22	2.47	0.411	53.34 KB
#6683	`WriteAndFlushEnrichedTraces`	net6.0	539μs	2.77μs	13μs	0.543	0	0	41.49 KB
#6683	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	668μs	3.39μs	15.5μs	0.332	0	0	41.73 KB
#6683	`WriteAndFlushEnrichedTraces`	net472	850μs	4.09μs	16.9μs	8.28	2.48	0.414	53.26 KB

Benchmarks.Trace.DbCommandBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6683

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0	1.137	1,442.46	1,269.20

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.44μs	2.09ns	8.1ns	0.0138	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.75μs	2.38ns	9.22ns	0.0139	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.07μs	4.15ns	16.1ns	0.156	0.00103	987 B
#6683	`ExecuteNonQuery`	net6.0	1.27μs	0.974ns	3.77ns	0.014	0	1.02 KB
#6683	`ExecuteNonQuery`	netcoreapp3.1	1.74μs	2.6ns	10.1ns	0.0139	0	1.02 KB
#6683	`ExecuteNonQuery`	net472	2.08μs	3.62ns	14ns	0.156	0.00104	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.21μs	0.404ns	1.51ns	0.0139	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.5μs	0.958ns	3.59ns	0.0128	976 B
master	`CallElasticsearch`	net472	2.46μs	2.36ns	9.14ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.37μs	1.01ns	3.76ns	0.013	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.71μs	0.965ns	3.61ns	0.0139	1.02 KB
master	`CallElasticsearchAsync`	net472	2.67μs	1.97ns	7.63ns	0.166	1.05 KB
#6683	`CallElasticsearch`	net6.0	1.27μs	1.58ns	5.9ns	0.0133	976 B
#6683	`CallElasticsearch`	netcoreapp3.1	1.56μs	0.634ns	2.45ns	0.0132	976 B
#6683	`CallElasticsearch`	net472	2.63μs	2.81ns	10.9ns	0.158	995 B
#6683	`CallElasticsearchAsync`	net6.0	1.35μs	0.639ns	2.39ns	0.0136	952 B
#6683	`CallElasticsearchAsync`	netcoreapp3.1	1.6μs	0.741ns	2.57ns	0.0137	1.02 KB
#6683	`CallElasticsearchAsync`	net472	2.64μs	1.24ns	4.64ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6683

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0	1.137	1,444.80	1,270.87

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.44μs	1.02ns	3.96ns	0.013	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.71μs	0.954ns	3.7ns	0.012	952 B
master	`ExecuteAsync`	net472	1.86μs	0.559ns	2.09ns	0.145	915 B
#6683	`ExecuteAsync`	net6.0	1.27μs	0.476ns	1.85ns	0.0133	952 B
#6683	`ExecuteAsync`	netcoreapp3.1	1.63μs	0.928ns	3.59ns	0.0131	952 B
#6683	`ExecuteAsync`	net472	1.91μs	0.575ns	2.07ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.41μs	1.19ns	4.47ns	0.0331	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.35μs	3.47ns	13ns	0.0374	2.85 KB
master	`SendAsync`	net472	7.37μs	2.03ns	7.85ns	0.495	3.12 KB
#6683	`SendAsync`	net6.0	4.42μs	1.91ns	7.4ns	0.031	2.31 KB
#6683	`SendAsync`	netcoreapp3.1	5.28μs	2.49ns	9.32ns	0.0368	2.85 KB
#6683	`SendAsync`	net472	7.47μs	2.58ns	10ns	0.493	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.58μs	0.493ns	1.78ns	0.023	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.19μs	1.07ns	3.99ns	0.0228	1.64 KB
master	`EnrichedLog`	net472	2.67μs	4.35ns	16.9ns	0.25	1.57 KB
#6683	`EnrichedLog`	net6.0	1.49μs	0.69ns	2.49ns	0.0232	1.64 KB
#6683	`EnrichedLog`	netcoreapp3.1	2.3μs	1.52ns	5.89ns	0.0217	1.64 KB
#6683	`EnrichedLog`	net472	2.65μs	3.6ns	13ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	112μs	212ns	822ns	0.0558	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	117μs	148ns	573ns	0.0582	0	4.28 KB
master	`EnrichedLog`	net472	151μs	82.1ns	307ns	0.681	0.227	4.46 KB
#6683	`EnrichedLog`	net6.0	114μs	247ns	958ns	0.056	0	4.28 KB
#6683	`EnrichedLog`	netcoreapp3.1	116μs	242ns	936ns	0.0584	0	4.28 KB
#6683	`EnrichedLog`	net472	153μs	481ns	1.86μs	0.673	0.224	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.11μs	0.496ns	1.72ns	0.0311	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.2μs	2.35ns	8.79ns	0.0294	2.2 KB
master	`EnrichedLog`	net472	5.03μs	1.01ns	3.79ns	0.32	2.02 KB
#6683	`EnrichedLog`	net6.0	2.92μs	0.928ns	3.47ns	0.0305	2.2 KB
#6683	`EnrichedLog`	netcoreapp3.1	4.07μs	1.31ns	4.88ns	0.0286	2.2 KB
#6683	`EnrichedLog`	net472	4.88μs	1.08ns	4.17ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.27μs	0.38ns	1.47ns	0.0159	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.77μs	0.486ns	1.68ns	0.015	1.14 KB
master	`SendReceive`	net472	2.12μs	2.85ns	11.1ns	0.183	1.16 KB
#6683	`SendReceive`	net6.0	1.41μs	1.22ns	4.72ns	0.0162	1.14 KB
#6683	`SendReceive`	netcoreapp3.1	1.77μs	1.02ns	3.96ns	0.0149	1.14 KB
#6683	`SendReceive`	net472	2.05μs	1.23ns	4.62ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.77μs	1.19ns	4.44ns	0.022	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.97μs	5.6ns	20.9ns	0.0217	1.65 KB
master	`EnrichedLog`	net472	4.3μs	3.26ns	12.6ns	0.323	2.04 KB
#6683	`EnrichedLog`	net6.0	2.74μs	0.833ns	3.12ns	0.0233	1.6 KB
#6683	`EnrichedLog`	netcoreapp3.1	3.82μs	1.86ns	7.19ns	0.021	1.65 KB
#6683	`EnrichedLog`	net472	4.39μs	2.78ns	10.8ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	391ns	0.461ns	1.79ns	0.00817	576 B
master	`StartFinishSpan`	netcoreapp3.1	557ns	0.82ns	3.18ns	0.00784	576 B
master	`StartFinishSpan`	net472	584ns	1.27ns	4.91ns	0.0915	578 B
master	`StartFinishScope`	net6.0	515ns	0.487ns	1.89ns	0.00983	696 B
master	`StartFinishScope`	netcoreapp3.1	742ns	1.68ns	6.07ns	0.00932	696 B
master	`StartFinishScope`	net472	839ns	2.27ns	8.78ns	0.105	658 B
#6683	`StartFinishSpan`	net6.0	402ns	0.171ns	0.662ns	0.00816	576 B
#6683	`StartFinishSpan`	netcoreapp3.1	564ns	0.633ns	2.37ns	0.00758	576 B
#6683	`StartFinishSpan`	net472	613ns	0.565ns	2.19ns	0.0917	578 B
#6683	`StartFinishScope`	net6.0	473ns	0.347ns	1.34ns	0.00986	696 B
#6683	`StartFinishScope`	netcoreapp3.1	735ns	0.534ns	2.07ns	0.0092	696 B
#6683	`StartFinishScope`	net472	852ns	0.621ns	2.4ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6683

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472	1.135	1,162.68	1,024.57

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	733ns	1.3ns	5.05ns	0.00951	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	907ns	1.48ns	5.73ns	0.00936	696 B
master	`RunOnMethodBegin`	net472	1.16μs	1.52ns	5.88ns	0.105	658 B
#6683	`RunOnMethodBegin`	net6.0	659ns	0.419ns	1.51ns	0.00983	696 B
#6683	`RunOnMethodBegin`	netcoreapp3.1	963ns	0.244ns	0.88ns	0.00928	696 B
#6683	`RunOnMethodBegin`	net472	1.02μs	0.542ns	2.1ns	0.104	658 B

andrewlock

Thanks for doing this!

andrewlock · 2025-02-19T09:02:14Z

docker-compose.yml

Yeah, generally we do, as it avoids a whole category of flake (network issue, API limits). It's not critical to do ahead of time though, and if we do the service bus one too, then we can update them at that point

andrewlock · 2025-02-19T09:03:35Z

docker-compose.yml

@@ -89,6 +89,17 @@ services:
    - "127.0.0.1:6391:6379"

 # Dependencies
+  cosmosdb-emulator:
+    image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest


The docs all specify the :vnext-preview tag rather than latest 🤔 I wonder if that's the source of the license issue you're seeing?

Suggested change

image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest

image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview

I'm very confused, I'll have to re-read it, the instructions in the larger doc that I followed are entirely different that the other one

andrewlock · 2025-02-19T09:05:30Z

docker-compose.yml

+      - AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=false
+      - AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1
+    ports:
+      - "8081:8081"


8081 is a kind of risky choice, as it's a commonly used port. We might want to publicly expose a different port and map it to the internal 8081, but we can leave it for now 👍

andrewlock · 2025-02-19T09:06:27Z

docker-compose.yml

+      - AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1
+    ports:
+      - "8081:8081"
+      - "10250-10255:10250-10255"


Do we really need all these ports 😕 The docs all say it's port 1234 we need to expose? 🤔 And that's only for the data explorer, so we probably don't even need to do that?

Suggested change

- "10250-10255:10250-10255"

andrewlock · 2025-02-19T09:08:15Z

docker-compose.yml

+    environment:
+      - AZURE_COSMOS_EMULATOR_PARTITION_COUNT=2
+      - AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=false
+      - AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=127.0.0.1


The docs say

The .NET and Java SDKs don't support HTTP mode in the emulator. Since this version of the emulator starts with HTTP by default, you will need to explicitly enable HTTPS when starting the container

So I think we need to set that? 🤔

andrewlock · 2025-02-19T09:08:55Z

docker-compose.yml

    environment:
      - TIMEOUT_LENGTH=120
-    command: servicestackredis:6379 stackexchangeredis:6379 stackexchangeredis-replica:6379 stackexchangeredis-single:6379 elasticsearch5:9200 elasticsearch6:9200 elasticsearch7:9200 sqlserver:1433 mongo:27017 postgres:5432 mysql:3306 mysql57:3306 rabbitmq:5672 kafka-broker:9092 kafka-zookeeper:2181 localstack:4566 couchbase:11210  # oracle:1521
+    command: servicestackredis:6379 stackexchangeredis:6379 stackexchangeredis-replica:6379 stackexchangeredis-single:6379 elasticsearch5:9200 elasticsearch6:9200 elasticsearch7:9200 sqlserver:1433 mongo:27017 postgres:5432 mysql:3306 mysql57:3306 rabbitmq:5672 kafka-broker:9092 kafka-zookeeper:2181 localstack:4566 couchbase:11210  cosmosbd-emulator:8081 # oracle:1521


If we do need to expose the data explorer port (e.g. 1234), we should include a check for it here

andrewlock · 2025-02-19T09:12:06Z

tracer/test/Datadog.Trace.ClrProfiler.IntegrationTests/CosmosTests.cs

@@ -34,13 +33,12 @@ public static IEnumerable<object[]> GetEnabledConfig()

        public override Result ValidateIntegrationSpan(MockSpan span, string metadataSchemaVersion) => span.IsCosmosDb(metadataSchemaVersion);

-        [SkippableTheory]
+        [Theory]
        [MemberData(nameof(GetEnabledConfig))]
        [Trait("Category", "EndToEnd")]
        [Trait("RunOnWindows", "True")]


Yeah, this gives a big flip. Currently the tests assume we're using the powershell-started cosmos emulator, so can only run on Windows. With this change, we can only run on Linux, so we need to remove the RunOnWindows config for now.

Also, you haven't configured the docker-compose for arm64 yet (there's separate testing and start-dependencies stages), so we'll need to do that.

Suggested change

[Trait("RunOnWindows", "True")]

andrewlock · 2025-02-19T09:12:57Z

tracer/test/test-applications/integrations/Samples.CosmosDb/Program.cs

-                    OpenTcpConnectionTimeout = TimeSpan.FromMinutes(1),
+                    // the docker container has a cert, we just ignore it, it isn't important
+                    // the docs say to do this!
+                    HttpClientFactory = () => new HttpClient(new HttpClientHandler()


FWIW, if this is a problem for older versions, I'm inclined to add an #if here, and just don't test those old versions in CI. It's still better than what we have now anyway.

bouwkast · 2025-02-19T13:54:05Z

It don't work and it is too much effort I'll just test locally

bouwkast added 2 commits February 18, 2025 17:21

Add cosmosdb-emulator to docker compose

031a379

Update CosmosDb sample to use docker emulator

313b57a

bouwkast requested review from a team as code owners February 18, 2025 22:32

bouwkast commented Feb 18, 2025

View reviewed changes

andrewlock reviewed Feb 19, 2025

View reviewed changes

bouwkast closed this Feb 19, 2025

Only test versions of Cosmos where we can disable SSL

ac7d84b

bouwkast reopened this Feb 19, 2025

	image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
	image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview

Setup CosmosDB Linux Docker container to run tests on CI #6683

Are you sure you want to change the base?

Setup CosmosDB Linux Docker container to run tests on CI #6683

Conversation

bouwkast commented Feb 18, 2025

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bouwkast commented Feb 18, 2025

andrewlock commented Feb 18, 2025 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Feb 18, 2025 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #6683

Raw results

Faster 🎉 in #6683

Raw results

Raw results

Faster 🎉 in #6683

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6683

Raw results

andrewlock left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bouwkast commented Feb 19, 2025

andrewlock commented Feb 18, 2025 •

edited

Loading

andrewlock commented Feb 18, 2025 •

edited

Loading