Skip to content

feat(amazonq): LSP telemetry/event messages trigger client side telemetry service calls #5511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
May 14, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
841947a
handle telemetry/event messages
samgst-amazon Mar 26, 2025
d660a3c
Update plugins/amazonq/shared/jetbrains-community/src/software/aws/to…
samgst-amazon Mar 26, 2025
c880e8d
Update plugins/amazonq/shared/jetbrains-community/src/software/aws/to…
samgst-amazon Mar 26, 2025
4831a5c
Merge branch 'feature/q-lsp' into samgst/q-lsp-telemetry
samgst-amazon Mar 26, 2025
085e456
parse metricUnit in util
samgst-amazon Mar 26, 2025
c3cd86f
Merge branch 'feature/q-lsp' into samgst/q-lsp-telemetry
samgst-amazon Mar 26, 2025
6d97ddc
warn for bad telemetryEvent case
samgst-amazon Mar 26, 2025
792b087
update test case
samgst-amazon Mar 26, 2025
0e22516
detekt
samgst-amazon Mar 27, 2025
37a0193
style
samgst-amazon Mar 27, 2025
e97eb33
style
samgst-amazon Mar 27, 2025
3219b41
Merge branch 'main' into samgst/q-lsp-telemetry
samgst-amazon Apr 18, 2025
e4309da
Merge branch 'main' into samgst/q-lsp-telemetry
rli May 8, 2025
f2507f2
Merge branch 'main' into samgst/q-lsp-telemetry
samgst-amazon May 9, 2025
e6083d3
Merge branch 'feature/q-lsp-chat' into samgst/q-lsp-telemetry
samgst-amazon May 9, 2025
f3f6f6f
merge error
samgst-amazon May 9, 2025
b698c0b
detekt
samgst-amazon May 9, 2025
c545757
handle mynah ui telemetry event
samgst-amazon May 9, 2025
a0881c2
detekt
samgst-amazon May 9, 2025
75c1a01
remove print
samgst-amazon May 9, 2025
666437e
send notification to server
samgst-amazon May 9, 2025
6b30379
detekt
samgst-amazon May 9, 2025
1058210
Merge branch 'feature/q-lsp-chat' into samgst/q-lsp-telemetry
samgst-amazon May 12, 2025
ea52630
fix data classes
samgst-amazon May 12, 2025
ac06230
remove createTime from datum builder
samgst-amazon May 13, 2025
ff7f920
detekt
samgst-amazon May 13, 2025
3652c52
Merge branch 'feature/q-lsp-chat' into samgst/q-lsp-telemetry
samgst-amazon May 13, 2025
cab6460
Merge branch 'feature/q-lsp-chat' into samgst/q-lsp-telemetry
samgst-amazon May 14, 2025
77457f2
Merge remote-tracking branch 'origin/samgst/q-lsp-telemetry' into sam…
samgst-amazon May 14, 2025
7d82d95
refactor lsp notification
samgst-amazon May 14, 2025
d8c3f12
remove unused data class
samgst-amazon May 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,55 @@
import org.eclipse.lsp4j.MessageType
import org.eclipse.lsp4j.PublishDiagnosticsParams
import org.eclipse.lsp4j.ShowMessageRequestParams
import software.amazon.awssdk.services.toolkittelemetry.model.MetricUnit
import software.aws.toolkits.core.utils.getLogger
import software.aws.toolkits.core.utils.warn
import software.aws.toolkits.jetbrains.core.credentials.AwsBearerTokenConnection
import software.aws.toolkits.jetbrains.core.credentials.ToolkitConnectionManager
import software.aws.toolkits.jetbrains.core.credentials.pinning.QConnection
import software.aws.toolkits.jetbrains.services.amazonq.lsp.model.aws.credentials.ConnectionMetadata
import software.aws.toolkits.jetbrains.services.amazonq.lsp.model.aws.credentials.SsoProfileData
import software.aws.toolkits.jetbrains.services.telemetry.TelemetryService
import software.aws.toolkits.jetbrains.settings.CodeWhispererSettings
import java.time.Instant
import java.util.concurrent.CompletableFuture

/**
* Concrete implementation of [AmazonQLanguageClient] to handle messages sent from server
*/
class AmazonQLanguageClientImpl(private val project: Project) : AmazonQLanguageClient {
override fun telemetryEvent(`object`: Any) {
println(`object`)
when (`object`) {

Check warning on line 32 in plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt

View check run for this annotation

Codecov / codecov/patch

plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt#L32

Added line #L32 was not covered by tests
is MutableMap<*, *> -> handleTelemetryMap(`object`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warn for the other case?

}
}

Check warning on line 35 in plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt

View check run for this annotation

Codecov / codecov/patch

plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt#L35

Added line #L35 was not covered by tests

private fun handleTelemetryMap(telemetryMap: MutableMap<*, *>) {
try {

Check warning on line 38 in plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt

View check run for this annotation

Codecov / codecov/patch

plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt#L38

Added line #L38 was not covered by tests
val name = telemetryMap["name"] as? String ?: return

@Suppress("UNCHECKED_CAST")

Check warning on line 41 in plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt

View check run for this annotation

Codecov / codecov/patch

plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt#L41

Added line #L41 was not covered by tests
val data = telemetryMap["data"] as? MutableMap<String, Any> ?: return

TelemetryService.getInstance().record(project) {
datum(name) {
createTime(Instant.now())

Check warning on line 46 in plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt

View check run for this annotation

Codecov / codecov/patch

plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt#L44-L46

Added lines #L44 - L46 were not covered by tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

time should be extracted from the metric?

unit(telemetryMap["unit"] as? MetricUnit ?: MetricUnit.NONE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does Flare model this event? i'm not sure if Gson will magically deserialize into these types

value(telemetryMap["value"] as? Double ?: 1.0)
passive(telemetryMap["passive"] as? Boolean ?: false)

telemetryMap["result"]?.let { result ->
metadata("result", result.toString())

Check warning on line 52 in plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt

View check run for this annotation

Codecov / codecov/patch

plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt#L52

Added line #L52 was not covered by tests
}

data.forEach { (key, value) ->
metadata(key, value.toString())
}
}
}
} catch (e: Exception) {
LOG.warn(e) { "Failed to process telemetry event: $telemetryMap" }

Check warning on line 61 in plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt

View check run for this annotation

Codecov / codecov/patch

plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt#L55-L61

Added lines #L55 - L61 were not covered by tests
}
}

override fun publishDiagnostics(diagnostics: PublishDiagnosticsParams) {
Expand Down Expand Up @@ -92,4 +127,8 @@
}
)
}

companion object {
private val LOG = getLogger<AmazonQLanguageClientImpl>()

Check warning on line 132 in plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt

View check run for this annotation

Codecov / codecov/patch

plugins/amazonq/shared/jetbrains-community/src/software/aws/toolkits/jetbrains/services/amazonq/lsp/AmazonQLanguageClientImpl.kt#L132

Added line #L132 was not covered by tests
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,23 +10,248 @@ import com.intellij.openapi.project.Project
import com.intellij.testFramework.ApplicationExtension
import io.mockk.every
import io.mockk.mockk
import io.mockk.mockkObject
import io.mockk.slot
import io.mockk.verify
import org.assertj.core.api.Assertions.assertThat
import org.assertj.core.api.Assertions.entry
import org.eclipse.lsp4j.ConfigurationItem
import org.eclipse.lsp4j.ConfigurationParams
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import software.amazon.awssdk.services.toolkittelemetry.model.MetricUnit
import software.aws.toolkits.core.telemetry.DefaultMetricEvent
import software.aws.toolkits.core.telemetry.MetricEvent
import software.aws.toolkits.jetbrains.core.credentials.AwsBearerTokenConnection
import software.aws.toolkits.jetbrains.core.credentials.ToolkitConnectionManager
import software.aws.toolkits.jetbrains.core.credentials.pinning.QConnection
import software.aws.toolkits.jetbrains.services.amazonq.lsp.model.aws.credentials.ConnectionMetadata
import software.aws.toolkits.jetbrains.services.amazonq.lsp.model.aws.credentials.SsoProfileData
import software.aws.toolkits.jetbrains.services.telemetry.TelemetryService
import software.aws.toolkits.jetbrains.settings.CodeWhispererSettings

@ExtendWith(ApplicationExtension::class)
class AmazonQLanguageClientImplTest {
private val project: Project = mockk(relaxed = true)
private val sut = AmazonQLanguageClientImpl(project)

@Test
fun `telemetryEvent handles basic event with name and data`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"data" to mapOf(
"key1" to "value1",
"key2" to 42
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.name).isEqualTo("test_event")
assertThat(datum.metadata).contains(
entry("key1", "value1"),
entry("key2", "42")
)
}

@Test
fun `telemetryEvent handles event with result field`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"result" to "success",
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.name).isEqualTo("test_event")
assertThat(datum.metadata).contains(
entry("key1", "value1"),
entry("result", "success")
)
}

@Test
fun `telemetryEvent uses custom unit value when provided`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"unit" to MetricUnit.MILLISECONDS,
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.unit).isEqualTo(MetricUnit.MILLISECONDS)
assertThat(datum.metadata).contains(entry("key1", "value1"))
}

@Test
fun `telemetryEvent uses custom value when provided`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"value" to 2.5,
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.value).isEqualTo(2.5)
assertThat(datum.metadata).contains(entry("key1", "value1"))
}

@Test
fun `telemetryEvent uses custom passive value when provided`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"passive" to true,
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.passive).isTrue()
assertThat(datum.metadata).contains(entry("key1", "value1"))
}

@Test
fun `telemetryEvent ignores event without name`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val event = mapOf(
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

verify(exactly = 0) {
telemetryService.record(project, any())
}
}

@Test
fun `telemetryEvent ignores event without data`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val event = mapOf(
"name" to "test_event"
)

sut.telemetryEvent(event)

verify(exactly = 0) {
telemetryService.record(project, any())
}
}

@Test
fun `telemetryEvent uses default values when not provided`() {
val telemetryService = mockk<TelemetryService>(relaxed = true)
mockkObject(TelemetryService)
every { TelemetryService.getInstance() } returns telemetryService

val builderCaptor = slot<MetricEvent.Builder.() -> Unit>()
every { telemetryService.record(project, capture(builderCaptor)) } returns Unit

val event = mapOf(
"name" to "test_event",
"data" to mapOf(
"key1" to "value1"
)
)

sut.telemetryEvent(event)

val builder = DefaultMetricEvent.builder()
builderCaptor.captured.invoke(builder)

val metricEvent = builder.build()
val datum = metricEvent.data.first()

assertThat(datum.unit).isEqualTo(MetricUnit.NONE)
assertThat(datum.value).isEqualTo(1.0)
assertThat(datum.passive).isFalse()
assertThat(datum.metadata).contains(entry("key1", "value1"))
}

@Test
fun `getConnectionMetadata returns connection metadata with start URL for bearer token connection`() {
val mockConnectionManager = mockk<ToolkitConnectionManager>()
Expand Down
Loading