Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logic to send analytics data #5545

Merged
merged 32 commits into from
Aug 20, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
34a361d
Modify telemetry functions
ChristopherDedominici Jul 22, 2024
24b35bb
add uuid npm package
ChristopherDedominici Jul 23, 2024
4407167
add a method in hh-core to get the telemetry directory
ChristopherDedominici Jul 23, 2024
40b00f5
add main logic (still a few minors TODOs)
ChristopherDedominici Jul 23, 2024
a664c3b
add tests
ChristopherDedominici Jul 23, 2024
648592c
small code reorganization
ChristopherDedominici Jul 24, 2024
e5d5d4a
Merge branch 'v-next' of github.com:NomicFoundation/hardhat into feat…
ChristopherDedominici Jul 24, 2024
94a3abe
Small PR comments' fixes
ChristopherDedominici Jul 24, 2024
c01024a
remove userType. CI is no longer sent
ChristopherDedominici Jul 29, 2024
170f773
remove old analytics client ids
ChristopherDedominici Jul 29, 2024
12a5cd6
Improve naming for functions
ChristopherDedominici Jul 29, 2024
89bbc8a
use builtin method to generate uuid
ChristopherDedominici Jul 29, 2024
f1ab5a4
remove unnecessary lambda functions from subprocess'file
ChristopherDedominici Jul 29, 2024
87f6ab4
use taskId and subtaskId instead of taskName and scopeName
ChristopherDedominici Jul 29, 2024
b4b96ed
use test keys fro google analytics
ChristopherDedominici Jul 29, 2024
6c7dcf9
remove subtaskId, task is an array of string
ChristopherDedominici Jul 29, 2024
52a7c1a
Merge branch 'v-next' of github.com:NomicFoundation/hardhat into feat…
ChristopherDedominici Jul 29, 2024
e3b2c98
Merge branch 'v-next' of github.com:NomicFoundation/hardhat into feat…
ChristopherDedominici Jul 29, 2024
2cb67d4
mark getTelemetryConsent as private and add send user telemetry consent
ChristopherDedominici Jul 30, 2024
12c36c4
small improve in logic and added more tests
ChristopherDedominici Jul 30, 2024
a3202f1
add a helper file for telemetry tests
ChristopherDedominici Aug 2, 2024
0fd6555
add ENV variables to simplify testing (e.g.: force to run in non inte…
ChristopherDedominici Aug 2, 2024
141174d
Add test explanation
ChristopherDedominici Aug 2, 2024
f1c7c7d
move the check on the ENV variable to force telemetry consent in tests
ChristopherDedominici Aug 5, 2024
3e59fa5
modify helper: wait for subprocess file to be readable
ChristopherDedominici Aug 5, 2024
39a1511
add logic to wait for the file to be readable
ChristopherDedominici Aug 5, 2024
b9df707
Smaller fixes based on the PR's comments
ChristopherDedominici Aug 6, 2024
093ae33
Merge branch 'v-next' of github.com:NomicFoundation/hardhat into feat…
ChristopherDedominici Aug 9, 2024
76c4422
add debug module
ChristopherDedominici Aug 9, 2024
ab5014b
add logs in telemetry-permission file
ChristopherDedominici Aug 9, 2024
e9ecacc
additional debug message
ChristopherDedominici Aug 9, 2024
082b407
Merge branch 'v-next' of github.com:NomicFoundation/hardhat into feat…
ChristopherDedominici Aug 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions v-next/core/src/global-dir.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,21 @@ export async function getCacheDir(): Promise<string> {
return cache;
}

/**
* Returns the path to the telemetry directory for the specified package.
* If no package name is provided, the default package name "hardhat" is used.
* Ensures that the directory exists before returning the path.
*
* @param packageName - The name of the package to get the telemetry directory for. Defaults to "hardhat".
*
* @returns A promise that resolves to the path of the telemetry directory.
*/
export async function getTelemetryDir(packageName?: string): Promise<string> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make sure that this dir is not deleted by the clean --global task? /cc @schaable

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, as the global flag deletes the cache path and telemetry uses the data path.

const { data } = await generatePaths(packageName);
await ensureDir(data);
return data;
}

async function generatePaths(packageName = "hardhat") {
const { default: envPaths } = await import("env-paths");
return envPaths(packageName);
Expand Down
122 changes: 122 additions & 0 deletions v-next/hardhat/src/internal/cli/telemetry/analytics/analytics.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
import type {
EventNames,
Payload,
TaskParams,
TelemetryConsentPayload,
} from "./types.js";

import os from "node:os";

import { spawnDetachedSubProcess } from "@ignored/hardhat-vnext-utils/subprocess";

import { getHardhatVersion } from "../../../utils/package.js";
import {
isTelemetryAllowedInEnvironment,
isTelemetryAllowed,
} from "../telemetry-permissions.js";

import { getAnalyticsClientId } from "./utils.js";

// TODO:log const log = debug("hardhat:core:global-dir");

const SESSION_ID = Math.random().toString();
const ENGAGEMENT_TIME_MSEC = "10000";

// Return a boolean for testing purposes to verify that analytics are not sent in CI environments
export async function sendTelemetryConsentAnalytics(
consent: boolean,
): Promise<boolean> {
// This is a special scenario where only the consent is sent, all the other analytics info
// (like node version, hardhat version, etc.) are stripped.

if (!isTelemetryAllowedInEnvironment()) {
return false;
}

const payload: TelemetryConsentPayload = {
client_id: "hardhat_telemetry_consent",
user_id: "hardhat_telemetry_consent",
user_properties: {},
events: [
{
name: "TelemetryConsentResponse",
params: {
userConsent: consent ? "yes" : "no",
},
},
],
};

await createSubprocessToSendAnalytics(payload);

return true;
}

export async function sendTaskAnalytics(taskId: string[]): Promise<boolean> {
const eventParams: TaskParams = {
task: taskId.join(", "),
};

return sendAnalytics("task", eventParams);
}

// Return a boolean for testing purposes to confirm whether analytics were sent based on the consent value and not in CI environments
async function sendAnalytics(
eventName: EventNames,
eventParams: TaskParams,
): Promise<boolean> {
if (!(await isTelemetryAllowed())) {
return false;
}

const payload = await buildPayload(eventName, eventParams);

await createSubprocessToSendAnalytics(payload);

return true;
}

async function createSubprocessToSendAnalytics(
payload: TelemetryConsentPayload | Payload,
): Promise<void> {
const fileExt =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's document this here with a comment because we'll forget it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done here, with other smaller fixes

process.env.HARDHAT_TEST_SUBPROCESS_RESULT_PATH !== undefined ? "ts" : "js";
const subprocessFile = `${import.meta.dirname}/subprocess.${fileExt}`;

const env: Record<string, string> = {};
if (process.env.HARDHAT_TEST_SUBPROCESS_RESULT_PATH !== undefined) {
// ATTENTION: only for testing
env.HARDHAT_TEST_SUBPROCESS_RESULT_PATH =
process.env.HARDHAT_TEST_SUBPROCESS_RESULT_PATH;
}

await spawnDetachedSubProcess(subprocessFile, [JSON.stringify(payload)], env);
}

async function buildPayload(
eventName: EventNames,
eventParams: TaskParams,
): Promise<Payload> {
const clientId = await getAnalyticsClientId();

return {
client_id: clientId,
user_id: clientId,
user_properties: {
projectId: { value: "hardhat-project" },
hardhatVersion: { value: await getHardhatVersion() },
operatingSystem: { value: os.platform() },
nodeVersion: { value: process.version },
},
events: [
{
name: eventName,
params: {
engagement_time_msec: ENGAGEMENT_TIME_MSEC,
session_id: SESSION_ID,
...eventParams,
},
},
],
};
}
27 changes: 27 additions & 0 deletions v-next/hardhat/src/internal/cli/telemetry/analytics/subprocess.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { writeJsonFile } from "@ignored/hardhat-vnext-utils/fs";
import { postJsonRequest } from "@ignored/hardhat-vnext-utils/request";

// These keys are expected to be public
const ANALYTICS_URL = "https://www.google-analytics.com/mp/collect";
const API_SECRET = "iXzTRik5RhahYpgiatSv1w";
const MEASUREMENT_ID = "G-ZFZWHGZ64H";

async function main(): Promise<void> {
const payload = JSON.parse(process.argv[2]);

if (process.env.HARDHAT_TEST_SUBPROCESS_RESULT_PATH === undefined) {
await postJsonRequest(ANALYTICS_URL, payload, {
queryParams: {
api_secret: API_SECRET,
measurement_id: MEASUREMENT_ID,
},
});

return;
}

// ATTENTION: only for testing
await writeJsonFile(process.env.HARDHAT_TEST_SUBPROCESS_RESULT_PATH, payload);
}

await main();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have top-level-await now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done here, with other smaller fixes

63 changes: 63 additions & 0 deletions v-next/hardhat/src/internal/cli/telemetry/analytics/types.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
export interface AnalyticsFile {
analytics: {
clientId: string;
};
}

/* eslint-disable @typescript-eslint/naming-convention -- these payload is formatted based on what google analytics expects*/
export interface BasePayload {
client_id: string;
user_id: string;
user_properties: {};
events: Array<{
name: string;
params: {
// From the GA docs: amount of time someone spends with your web
// page in focus or app screen in the foreground.
// The parameter has no use for our app, but it's required in order
// for user activity to display in standard reports like Realtime.
engagement_time_msec?: string;
session_id?: string;
};
}>;
}

export interface TelemetryConsentPayload extends BasePayload {
events: Array<{
name: "TelemetryConsentResponse";
params: {
userConsent: "yes" | "no";
session_id?: string;
};
}>;
}

export type EventNames = "task";

export interface TaskParams {
task: string;
}

export interface Payload extends BasePayload {
user_properties: {
projectId: {
value: string;
};
hardhatVersion: {
value: string;
};
operatingSystem: {
value: string;
};
nodeVersion: {
value: string;
};
};
events: Array<{
name: EventNames;
params: {
engagement_time_msec: string;
session_id: string;
} & TaskParams;
}>;
}
54 changes: 54 additions & 0 deletions v-next/hardhat/src/internal/cli/telemetry/analytics/utils.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
import type { AnalyticsFile } from "./types.js";

import path from "node:path";

import { getTelemetryDir } from "@ignored/hardhat-vnext-core/global-dir";
import {
exists,
readJsonFile,
writeJsonFile,
} from "@ignored/hardhat-vnext-utils/fs";

const ANALYTICS_FILE_NAME = "analytics.json";

export async function getAnalyticsClientId(): Promise<string> {
let clientId = await readAnalyticsClientId();

if (clientId === undefined) {
// TODO:log log("Client Id not found, generating a new one");
clientId = crypto.randomUUID();

await writeAnalyticsClientId(clientId);
}

return clientId;
}

async function readAnalyticsClientId(): Promise<string | undefined> {
const globalTelemetryDir = await getTelemetryDir();
const filePath = path.join(globalTelemetryDir, ANALYTICS_FILE_NAME);

// TODO:log log(`Looking up Client Id at ${filePath}`);

if ((await exists(filePath)) === false) {
return undefined;
}

const data: AnalyticsFile = await readJsonFile(filePath);
const clientId = data.analytics.clientId;
// TODO:log log(`Client Id found: ${clientId}`);

return clientId;
}

async function writeAnalyticsClientId(clientId: string): Promise<void> {
const globalTelemetryDir = await getTelemetryDir();
const filePath = path.join(globalTelemetryDir, ANALYTICS_FILE_NAME);
await writeJsonFile(filePath, {
analytics: {
clientId,
},
});

// TODO:log log(`Stored clientId ${clientId}`);
}
54 changes: 30 additions & 24 deletions v-next/hardhat/src/internal/cli/telemetry/telemetry-permissions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import {

import { confirmationPromptWithTimeout } from "../prompt/prompt.js";

import { sendTelemetryConsentAnalytics } from "./analytics/analytics.js";

interface TelemetryConsent {
consent: boolean;
}
Expand Down Expand Up @@ -45,17 +47,43 @@ export async function isTelemetryAllowed(): Promise<boolean> {
return false;
}

// ATTENTION: only for testing
if (process.env.HARDHAT_TEST_TELEMETRY_CONSENT_VALUE !== undefined) {
return process.env.HARDHAT_TEST_TELEMETRY_CONSENT_VALUE === "true"
? true
: false;
}

const consent = await getTelemetryConsent();

return consent !== undefined ? consent : false;
}

/**
* Determines if telemetry is allowed in the current environment.
* This function checks various environmental factors to decide if telemetry data can be collected.
* It verifies that the environment is not a continuous integration (CI) environment, that the terminal is interactive,
* and that telemetry has not been explicitly disabled through an environment variable.
*
* @returns True if telemetry is allowed in the environment, false otherwise.
*/
export function isTelemetryAllowedInEnvironment(): boolean {
return (
(!isCi() &&
process.stdout.isTTY === true &&
process.env.HARDHAT_DISABLE_TELEMETRY_PROMPT !== "true") ||
// ATTENTION: used in tests to force telemetry execution
process.env.HARDHAT_TEST_INTERACTIVE_ENV === "true"
);
}

/**
* Retrieves the user's telemetry consent status from the consent file.
*
* @returns True if the user consents to telemetry, false if they do not consent,
* and undefined if no consent has been provided.
*/
export async function getTelemetryConsent(): Promise<boolean | undefined> {
async function getTelemetryConsent(): Promise<boolean | undefined> {
const telemetryConsentFilePath = await getTelemetryConsentFilePath();

if (await exists(telemetryConsentFilePath)) {
Expand All @@ -67,23 +95,6 @@ export async function getTelemetryConsent(): Promise<boolean | undefined> {
return undefined;
}

/**
* Determines if telemetry is allowed in the current environment.
* This function checks various environmental factors to decide if telemetry data can be collected.
* It verifies that the environment is not a continuous integration (CI) environment, that the terminal is interactive,
* and that telemetry has not been explicitly disabled through an environment variable.
*
* @returns True if telemetry is allowed in the environment, false otherwise.
*/
export function isTelemetryAllowedInEnvironment(): boolean {
return (
(!isCi() &&
process.stdout.isTTY === true &&
process.env.HARDHAT_DISABLE_TELEMETRY_PROMPT !== "true") ||
process.env.HARDHAT_ENABLE_TELEMETRY_IN_TEST === "true" // Used in tests to force telemetry execution
);
}

async function getTelemetryConsentFilePath() {
const configDir = await getConfigDir();
return path.join(configDir, "telemetry-consent.json");
Expand All @@ -99,12 +110,7 @@ async function requestTelemetryConsent(): Promise<boolean> {
// Store user's consent choice
await writeJsonFile(await getTelemetryConsentFilePath(), { consent });

// TODO: this will be enabled in a following PR as soon as the function to send telemetry is implemented
// const subprocessFilePath = path.join(
// path.dirname(fileURLToPath(import.meta.url)),
// "report-telemetry-consent.js",
// );
// await spawnDetachedSubProcess(subprocessFilePath, [consent ? "yes" : "no"]);
await sendTelemetryConsentAnalytics(consent);

return consent;
}
Expand Down
Loading