Skip to content

feat(amazonq): LSP telemetry/event messages trigger client side telemetry service calls #5511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
May 14, 2025
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
841947a
handle telemetry/event messages
samgst-amazon Mar 26, 2025
d660a3c
Update plugins/amazonq/shared/jetbrains-community/src/software/aws/to…
samgst-amazon Mar 26, 2025
c880e8d
Update plugins/amazonq/shared/jetbrains-community/src/software/aws/to…
samgst-amazon Mar 26, 2025
4831a5c
Merge branch 'feature/q-lsp' into samgst/q-lsp-telemetry
samgst-amazon Mar 26, 2025
085e456
parse metricUnit in util
samgst-amazon Mar 26, 2025
c3cd86f
Merge branch 'feature/q-lsp' into samgst/q-lsp-telemetry
samgst-amazon Mar 26, 2025
6d97ddc
warn for bad telemetryEvent case
samgst-amazon Mar 26, 2025
792b087
update test case
samgst-amazon Mar 26, 2025
0e22516
detekt
samgst-amazon Mar 27, 2025
37a0193
style
samgst-amazon Mar 27, 2025
e97eb33
style
samgst-amazon Mar 27, 2025
3219b41
Merge branch 'main' into samgst/q-lsp-telemetry
samgst-amazon Apr 18, 2025
e4309da
Merge branch 'main' into samgst/q-lsp-telemetry
rli May 8, 2025
f2507f2
Merge branch 'main' into samgst/q-lsp-telemetry
samgst-amazon May 9, 2025
e6083d3
Merge branch 'feature/q-lsp-chat' into samgst/q-lsp-telemetry
samgst-amazon May 9, 2025
f3f6f6f
merge error
samgst-amazon May 9, 2025
b698c0b
detekt
samgst-amazon May 9, 2025
c545757
handle mynah ui telemetry event
samgst-amazon May 9, 2025
a0881c2
detekt
samgst-amazon May 9, 2025
75c1a01
remove print
samgst-amazon May 9, 2025
666437e
send notification to server
samgst-amazon May 9, 2025
6b30379
detekt
samgst-amazon May 9, 2025
1058210
Merge branch 'feature/q-lsp-chat' into samgst/q-lsp-telemetry
samgst-amazon May 12, 2025
ea52630
fix data classes
samgst-amazon May 12, 2025
ac06230
remove createTime from datum builder
samgst-amazon May 13, 2025
ff7f920
detekt
samgst-amazon May 13, 2025
3652c52
Merge branch 'feature/q-lsp-chat' into samgst/q-lsp-telemetry
samgst-amazon May 13, 2025
cab6460
Merge branch 'feature/q-lsp-chat' into samgst/q-lsp-telemetry
samgst-amazon May 14, 2025
77457f2
Merge remote-tracking branch 'origin/samgst/q-lsp-telemetry' into sam…
samgst-amazon May 14, 2025
7d82d95
refactor lsp notification
samgst-amazon May 14, 2025
d8c3f12
remove unused data class
samgst-amazon May 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,58 @@
import org.eclipse.lsp4j.MessageType
import org.eclipse.lsp4j.PublishDiagnosticsParams
import org.eclipse.lsp4j.ShowMessageRequestParams
import software.amazon.awssdk.services.toolkittelemetry.model.MetricUnit

Check warning on line 14 in plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt

View workflow job for this annotation

GitHub Actions / Qodana Community for JVM

Unused import directive

Unused import directive
import software.aws.toolkits.core.utils.getLogger
import software.aws.toolkits.core.utils.warn
import software.aws.toolkits.jetbrains.core.credentials.AwsBearerTokenConnection
import software.aws.toolkits.jetbrains.core.credentials.ToolkitConnectionManager
import software.aws.toolkits.jetbrains.core.credentials.pinning.QConnection
import software.aws.toolkits.jetbrains.services.amazonq.lsp.model.aws.credentials.ConnectionMetadata
import software.aws.toolkits.jetbrains.services.amazonq.lsp.model.aws.credentials.SsoProfileData
import software.aws.toolkits.jetbrains.services.amazonq.lsp.util.TelemetryParsingUtil
import software.aws.toolkits.jetbrains.services.telemetry.TelemetryService
import software.aws.toolkits.jetbrains.settings.CodeWhispererSettings
import java.time.Instant
import java.util.concurrent.CompletableFuture

/**
* Concrete implementation of [AmazonQLanguageClient] to handle messages sent from server
*/
class AmazonQLanguageClientImpl(private val project: Project) : AmazonQLanguageClient {

private fun handleTelemetryMap(telemetryMap: Map<*, *>) {
try {
val name = telemetryMap["name"] as? String ?: return

@Suppress("UNCHECKED_CAST")
val data = telemetryMap["data"] as? Map<String, Any> ?: return
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this the correct action here? are there any telemetry events we want to be emitting that would have no data?


TelemetryService.getInstance().record(project) {
datum(name) {
createTime(Instant.now())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

time should be extracted from the metric?

unit(TelemetryParsingUtil.parseMetricUnit(telemetryMap["unit"]))
value(telemetryMap["value"] as? Double ?: 1.0)
passive(telemetryMap["passive"] as? Boolean ?: false)

telemetryMap["result"]?.let { result ->
metadata("result", result.toString())
}

data.forEach { (key, value) ->
metadata(key, value.toString())
}
}
}
} catch (e: Exception) {
LOG.warn(e) { "Failed to process telemetry event: $telemetryMap" }
}
}

override fun telemetryEvent(`object`: Any) {
println(`object`)
when (`object`) {
is Map<*, *> -> handleTelemetryMap(`object`)
else -> LOG.warn { "Unexpected telemetry event: $`object`" }
}
}

override fun publishDiagnostics(diagnostics: PublishDiagnosticsParams) {
Expand Down Expand Up @@ -92,4 +130,8 @@
}
)
}

companion object {
private val LOG = getLogger<AmazonQLanguageClientImpl>()
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Copyright 2025 Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0

package software.aws.toolkits.jetbrains.services.amazonq.lsp.util

import software.amazon.awssdk.services.toolkittelemetry.model.MetricUnit

object TelemetryParsingUtil {

fun parseMetricUnit(value: Any?): MetricUnit {
return when (value) {
is String -> MetricUnit.fromValue(value) ?: MetricUnit.NONE
is MetricUnit -> value
else -> MetricUnit.NONE
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,28 +5,296 @@
package software.aws.toolkits.jetbrains.services.amazonq.lsp

import com.google.gson.Gson
import com.google.gson.GsonBuilder
import com.google.gson.ToNumberPolicy
import com.intellij.openapi.components.service
import com.intellij.openapi.project.Project
import com.intellij.testFramework.ApplicationExtension
import io.mockk.every
import io.mockk.mockk
import io.mockk.mockkObject
import io.mockk.slot
import io.mockk.verify
import org.assertj.core.api.Assertions.assertThat
import org.assertj.core.api.Assertions.entry
import org.eclipse.lsp4j.ConfigurationItem
import org.eclipse.lsp4j.ConfigurationParams
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import software.amazon.awssdk.services.toolkittelemetry.model.MetricUnit
import software.aws.toolkits.core.telemetry.DefaultMetricEvent
import software.aws.toolkits.core.telemetry.MetricEvent
import software.aws.toolkits.jetbrains.core.credentials.AwsBearerTokenConnection
import software.aws.toolkits.jetbrains.core.credentials.ToolkitConnectionManager
import software.aws.toolkits.jetbrains.core.credentials.pinning.QConnection
import software.aws.toolkits.jetbrains.services.amazonq.lsp.model.aws.credentials.ConnectionMetadata
import software.aws.toolkits.jetbrains.services.amazonq.lsp.model.aws.credentials.SsoProfileData
import software.aws.toolkits.jetbrains.services.telemetry.TelemetryService
import software.aws.toolkits.jetbrains.settings.CodeWhispererSettings

@ExtendWith(ApplicationExtension::class)
class AmazonQLanguageClientImplTest {
private val project: Project = mockk(relaxed = true)
private val sut = AmazonQLanguageClientImpl(project)

@Test
fun `telemetryEvent handles basic event with name and data`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"data" to mapOf(
"key1" to "value1",
"key2" to 42
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.name).isEqualTo("test_event")
assertThat(datum.metadata).contains(
entry("key1", "value1"),
entry("key2", "42")
)
}

@Test
fun `telemetryEvent handles event with result field`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"result" to "success",
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.name).isEqualTo("test_event")
assertThat(datum.metadata).contains(
entry("key1", "value1"),
entry("result", "success")
)
}

@Test
fun `telemetryEvent uses custom unit value when provided`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"unit" to "Bytes",
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.unit).isEqualTo(MetricUnit.BYTES)
assertThat(datum.metadata).contains(entry("key1", "value1"))
}

@Test
fun `telemetryEvent uses custom value when provided`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"value" to 2.5,
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.value).isEqualTo(2.5)
assertThat(datum.metadata).contains(entry("key1", "value1"))
}

@Test
fun `telemetryEvent uses custom passive value when provided`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"passive" to true,
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.passive).isTrue()
assertThat(datum.metadata).contains(entry("key1", "value1"))
}

@Test
fun `telemetryEvent ignores event without name`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val event = mapOf(
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

verify(exactly = 0) {
telemetryService.record(project, any())
}
}

@Test
fun `telemetryEvent ignores event without data`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val event = mapOf(
"name" to "test_event"
)

sut.telemetryEvent(event)

verify(exactly = 0) {
telemetryService.record(project, any())
}
}

@Test
fun `telemetryEvent uses default values when not provided`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.unit).isEqualTo(MetricUnit.NONE)
assertThat(datum.value).isEqualTo(1.0)
assertThat(datum.passive).isFalse()
assertThat(datum.metadata).contains(entry("key1", "value1"))
}

@Test
fun `test GSON deserialization behavior for telemetryEvent`() {
val gson = GsonBuilder()
.setObjectToNumberStrategy(ToNumberPolicy.LONG_OR_DOUBLE)
.create()

val jsonString = """
{
"name": "test_event",
"value": 3.0,
"passive": true,
"unit": "Milliseconds",
"data": {
"key1": "value1"
}
}
""".trimIndent()

val result = gson.fromJson(jsonString, Map::class.java)

val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

sut.telemetryEvent(result)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.passive).isTrue()
assertThat(datum.unit).isEqualTo(MetricUnit.MILLISECONDS)
assertThat(datum.value).isEqualTo(3.0)
assertThat(datum.metadata).contains(entry("key1", "value1"))
}

@Test
fun `getConnectionMetadata returns connection metadata with start URL for bearer token connection`() {
val mockConnectionManager = mockk<ToolkitConnectionManager>()
Expand Down
Loading