-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
feat(node): Add ClickHouse client OpenTelemetry instrumentation #18625
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mdhamed238
wants to merge
5
commits into
getsentry:develop
Choose a base branch
from
mdhamed238:feat/clickhouse-integration
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
4b6b56a
feat(node): add ClickHouse client instrumentation
mdhamed238 14c0600
fix(node): ensure ClickHouse spans have sentry.origin on error paths …
mdhamed238 5b7755f
fix(node): pass actual response to ClickHouse responseHook and handle
mdhamed238 8cbc18e
test(node): use it.each for ClickHouse SQL operation tests (#15966)
mdhamed238 568eedd
fix(node): wrap ClickHouse responseHook calls in try-catch to prevent
mdhamed238 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
40 changes: 40 additions & 0 deletions
40
packages/node/src/integrations/tracing/clickhouse/clickhouse.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| import type { Span } from '@opentelemetry/api'; | ||
| import type { IntegrationFn } from '@sentry/core'; | ||
| import { defineIntegration } from '@sentry/core'; | ||
| import { addOriginToSpan, generateInstrumentOnce } from '@sentry/node-core'; | ||
| import { ClickHouseInstrumentation } from './instrumentation' | ||
|
|
||
| const INTEGRATION_NAME = 'Clickhouse'; | ||
|
|
||
| export const instrumentClickhouse = generateInstrumentOnce( | ||
| INTEGRATION_NAME, | ||
| () => | ||
| new ClickHouseInstrumentation({ | ||
| responseHook(span: Span) { | ||
| addOriginToSpan(span, 'auto.db.otel.clickhouse'); | ||
| }, | ||
| }), | ||
| ); | ||
|
|
||
| const _clickhouseIntegration = (() => { | ||
| return { | ||
| name: INTEGRATION_NAME, | ||
| setupOnce() { | ||
| instrumentClickhouse(); | ||
| }, | ||
| }; | ||
| }) satisfies IntegrationFn; | ||
|
|
||
| /** | ||
| * Adds Sentry tracing instrumentation for the [ClickHouse](https://www.npmjs.com/package/@clickhouse/client) library. | ||
| * | ||
| * @example | ||
| * ```javascript | ||
| * const Sentry = require('@sentry/node'); | ||
| * | ||
| * Sentry.init({ | ||
| * integrations: [Sentry.clickhouseIntegration()], | ||
| * }); | ||
| * ``` | ||
| */ | ||
| export const clickhouseIntegration = defineIntegration(_clickhouseIntegration); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| export { clickhouseIntegration, instrumentClickhouse } from './clickhouse'; | ||
| export { ClickHouseInstrumentation } from './instrumentation'; | ||
| export type { ClickHouseInstrumentationConfig } from './types'; |
51 changes: 51 additions & 0 deletions
51
packages/node/src/integrations/tracing/clickhouse/instrumentation.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| import type { InstrumentationModuleDefinition } from '@opentelemetry/instrumentation'; | ||
| import { InstrumentationBase, InstrumentationNodeModuleDefinition } from '@opentelemetry/instrumentation'; | ||
| import { SDK_VERSION } from '@sentry/core'; | ||
| import { type ClickHouseModuleExports,patchClickHouseClient } from './patch'; | ||
| import type { ClickHouseInstrumentationConfig } from './types'; | ||
|
|
||
| const PACKAGE_NAME = '@sentry/instrumentation-clickhouse'; | ||
| const supportedVersions = ['>=0.0.1']; | ||
|
|
||
| /** | ||
| * | ||
| */ | ||
| export class ClickHouseInstrumentation extends InstrumentationBase<ClickHouseInstrumentationConfig> { | ||
| public constructor(config: ClickHouseInstrumentationConfig = {}) { | ||
| super(PACKAGE_NAME, SDK_VERSION, config); | ||
| } | ||
|
|
||
| /** | ||
| * | ||
| */ | ||
| public override init(): InstrumentationModuleDefinition { | ||
| return new InstrumentationNodeModuleDefinition( | ||
| '@clickhouse/client', | ||
| supportedVersions, | ||
| moduleExports => | ||
| patchClickHouseClient(moduleExports as ClickHouseModuleExports, { | ||
| wrap: this._wrap.bind(this), | ||
| unwrap: this._unwrap.bind(this), | ||
| tracer: this.tracer, | ||
| getConfig: this.getConfig.bind(this), | ||
| isEnabled: this.isEnabled.bind(this), | ||
| }), | ||
| moduleExports => { | ||
| const moduleExportsTyped = moduleExports as ClickHouseModuleExports; | ||
| const ClickHouseClient = moduleExportsTyped.ClickHouseClient; | ||
| if (ClickHouseClient && typeof ClickHouseClient === 'function' && 'prototype' in ClickHouseClient) { | ||
| const ClickHouseClientCtor = ClickHouseClient as new () => { | ||
| query: unknown; | ||
| insert: unknown; | ||
| exec: unknown; | ||
| command: unknown; | ||
| }; | ||
| this._unwrap(ClickHouseClientCtor.prototype, 'query'); | ||
| this._unwrap(ClickHouseClientCtor.prototype, 'insert'); | ||
| this._unwrap(ClickHouseClientCtor.prototype, 'exec'); | ||
| this._unwrap(ClickHouseClientCtor.prototype, 'command'); | ||
| } | ||
| }, | ||
| ); | ||
| } | ||
| } |
233 changes: 233 additions & 0 deletions
233
packages/node/src/integrations/tracing/clickhouse/patch.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,233 @@ | ||
| import type { Tracer } from '@opentelemetry/api'; | ||
| import { context, SpanKind, SpanStatusCode, trace } from '@opentelemetry/api'; | ||
| import type { InstrumentationBase } from '@opentelemetry/instrumentation'; | ||
| import { SEMANTIC_ATTRIBUTE_SENTRY_OP } from '@sentry/core'; | ||
| import type { ClickHouseInstrumentationConfig } from './types'; | ||
| import { addExecutionStats, extractOperation, extractSummary, sanitizeQueryText } from './utils'; | ||
|
|
||
| export interface ClickHouseModuleExports { | ||
| ClickHouseClient: unknown; | ||
| } | ||
|
|
||
| export interface PatchClickHouseOptions { | ||
| getConfig: () => ClickHouseInstrumentationConfig; | ||
| isEnabled: () => boolean; | ||
| tracer: Tracer; | ||
| unwrap: InstrumentationBase<ClickHouseInstrumentationConfig>['_unwrap']; | ||
| wrap: InstrumentationBase<ClickHouseInstrumentationConfig>['_wrap']; | ||
| } | ||
|
|
||
| // ClickHouse-specific semantic attributes | ||
| const SEMATTRS_DB_SYSTEM = 'db.system'; | ||
| const SEMATTRS_DB_OPERATION = 'db.operation'; | ||
| const SEMATTRS_DB_STATEMENT = 'db.statement'; | ||
| const SEMATTRS_DB_NAME = 'db.name'; | ||
| const SEMATTRS_NET_PEER_NAME = 'net.peer.name'; | ||
| const SEMATTRS_NET_PEER_PORT = 'net.peer.port'; | ||
|
|
||
| // Type definitions for ClickHouse client internals | ||
| interface ClickHouseClientInstance { | ||
| query: unknown; | ||
| insert: unknown; | ||
| exec: unknown; | ||
| command: unknown; | ||
| connection_params?: { url?: string }; | ||
| options?: { url?: string }; | ||
| } | ||
|
|
||
| interface ClickHouseQueryParams { | ||
| [key: string]: unknown; | ||
| query?: string; | ||
| } | ||
|
|
||
| interface ClickHouseInsertParams { | ||
| [key: string]: unknown; | ||
| table?: string; | ||
| format?: string; | ||
| columns?: string[] | { except?: string[] }; | ||
| } | ||
|
|
||
| interface ClickHouseResponse { | ||
| [key: string]: unknown; | ||
| response_headers?: Record<string, unknown>; | ||
| headers?: Record<string, unknown>; | ||
| } | ||
|
|
||
| /** | ||
| * Patches the ClickHouse client to add OpenTelemetry instrumentation. | ||
| */ | ||
| export function patchClickHouseClient( | ||
| moduleExports: ClickHouseModuleExports, | ||
| options: PatchClickHouseOptions, | ||
| ): ClickHouseModuleExports { | ||
| const { wrap, tracer, getConfig, isEnabled } = options; | ||
| const ClickHouseClient = moduleExports.ClickHouseClient; | ||
|
|
||
| if (!ClickHouseClient || typeof ClickHouseClient !== 'function' || !('prototype' in ClickHouseClient)) { | ||
| return moduleExports; | ||
| } | ||
|
|
||
| const ClickHouseClientCtor = ClickHouseClient as new () => { | ||
| query: unknown; | ||
| insert: unknown; | ||
| exec: unknown; | ||
| command: unknown; | ||
| }; | ||
| const prototype = ClickHouseClientCtor.prototype; | ||
|
|
||
| const patchGeneric = (methodName: string): void => { | ||
| wrap( | ||
| prototype, | ||
| methodName, | ||
| createPatchHandler(methodName, tracer, getConfig, isEnabled, args => { | ||
| const params = (args[0] || {}) as ClickHouseQueryParams; | ||
| const queryText = params.query || (typeof params === 'string' ? params : ''); | ||
| return { queryText }; | ||
| }), | ||
| ); | ||
| }; | ||
|
|
||
| const patchInsert = (): void => { | ||
| wrap( | ||
| prototype, | ||
| 'insert', | ||
| createPatchHandler('insert', tracer, getConfig, isEnabled, args => { | ||
| const params = (args[0] || {}) as ClickHouseInsertParams; | ||
| const table = params.table || '<unknown>'; | ||
| const format = params.format || 'JSONCompactEachRow'; | ||
| let statement = `INSERT INTO ${table}`; | ||
| if (params.columns) { | ||
| if (Array.isArray(params.columns)) { | ||
| statement += ` (${params.columns.join(', ')})`; | ||
| } else if (params.columns.except) { | ||
| statement += ` (* EXCEPT (${params.columns.except.join(', ')}))`; | ||
| } | ||
| } | ||
| statement += ` FORMAT ${format}`; | ||
| return { queryText: statement, operation: 'INSERT' }; | ||
| }), | ||
| ); | ||
| }; | ||
|
|
||
| patchGeneric('query'); | ||
| patchGeneric('exec'); | ||
| patchGeneric('command'); | ||
| patchInsert(); | ||
|
|
||
| return moduleExports; | ||
| } | ||
|
|
||
| function createPatchHandler( | ||
| methodName: string, | ||
| tracer: Tracer, | ||
| getConfig: () => ClickHouseInstrumentationConfig, | ||
| isEnabled: () => boolean, | ||
| attributesExtractor: (args: unknown[]) => { queryText: string; operation?: string }, | ||
| ) { | ||
| return function (original: (...args: unknown[]) => unknown) { | ||
| return function (this: ClickHouseClientInstance, ...args: unknown[]): unknown { | ||
| if (!isEnabled()) { | ||
| return original.apply(this, args); | ||
| } | ||
|
|
||
| const config = getConfig(); | ||
| let extraction; | ||
| try { | ||
| extraction = attributesExtractor(args); | ||
| } catch { | ||
| extraction = { queryText: '' }; | ||
| } | ||
|
|
||
| const { queryText, operation: explicitOp } = extraction; | ||
| const operation = explicitOp || (queryText ? extractOperation(queryText) : methodName.toUpperCase()); | ||
| const spanName = operation ? `${operation} clickhouse` : `${methodName} clickhouse`; | ||
|
|
||
| const span = tracer.startSpan(spanName, { | ||
| kind: SpanKind.CLIENT, | ||
| attributes: { | ||
| [SEMATTRS_DB_SYSTEM]: 'clickhouse', | ||
| [SEMANTIC_ATTRIBUTE_SENTRY_OP]: 'db.query', | ||
| [SEMATTRS_DB_OPERATION]: operation, | ||
| }, | ||
| }); | ||
|
|
||
| if (config.dbName) { | ||
| span.setAttribute(SEMATTRS_DB_NAME, config.dbName); | ||
| } | ||
| if (config.captureQueryText !== false && queryText) { | ||
| const maxLength = config.maxQueryLength || 1000; | ||
| span.setAttribute(SEMATTRS_DB_STATEMENT, sanitizeQueryText(queryText, maxLength)); | ||
| } | ||
| if (config.peerName) { | ||
| span.setAttribute(SEMATTRS_NET_PEER_NAME, config.peerName); | ||
| } | ||
| if (config.peerPort) { | ||
| span.setAttribute(SEMATTRS_NET_PEER_PORT, config.peerPort); | ||
| } | ||
| if (!config.peerName || !config.peerPort) { | ||
| try { | ||
| const clientConfig = this.connection_params || this.options; | ||
| if (clientConfig?.url) { | ||
| const url = new URL(clientConfig.url); | ||
| if (!config.peerName) { | ||
| span.setAttribute(SEMATTRS_NET_PEER_NAME, url.hostname); | ||
| } | ||
| if (!config.peerPort) { | ||
| span.setAttribute(SEMATTRS_NET_PEER_PORT, parseInt(url.port, 10) || 8123); | ||
| } | ||
| } | ||
| } catch { | ||
| // ignore failures in auto-discovery | ||
| } | ||
| } | ||
|
|
||
| return context.with(trace.setSpan(context.active(), span), () => { | ||
| const onSuccess = (response: ClickHouseResponse): ClickHouseResponse => { | ||
| if (config.captureExecutionStats !== false && response) { | ||
| const headers = response.response_headers || response.headers; | ||
| if (headers) { | ||
| const summary = extractSummary(headers); | ||
| if (summary) { | ||
| addExecutionStats(span, summary); | ||
| } | ||
| } | ||
| } | ||
| if (config.responseHook) { | ||
| try { | ||
| config.responseHook(span, response); | ||
| } catch { | ||
| // Ignore errors from user-provided hooks | ||
| } | ||
| } | ||
cursor[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| span.setStatus({ code: SpanStatusCode.OK }); | ||
| span.end(); | ||
| return response; | ||
| }; | ||
|
|
||
| const onError = (error: Error): never => { | ||
| if (config.responseHook) { | ||
| try { | ||
| config.responseHook(span, undefined); | ||
| } catch { | ||
| // Ignore errors from user-provided hooks | ||
| } | ||
| } | ||
| span.recordException(error); | ||
| span.setStatus({ code: SpanStatusCode.ERROR, message: error.message }); | ||
| span.end(); | ||
| throw error; | ||
cursor[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| }; | ||
|
|
||
| try { | ||
| const result = original.apply(this, args) as unknown; | ||
| if (result && typeof result === 'object' && 'then' in result && typeof result.then === 'function') { | ||
| return (result as Promise<ClickHouseResponse>).then(onSuccess, onError); | ||
| } | ||
| return onSuccess(result as ClickHouseResponse); | ||
| } catch (error) { | ||
| return onError(error as Error); | ||
| } | ||
| }); | ||
| }; | ||
| }; | ||
| } | ||
45 changes: 45 additions & 0 deletions
45
packages/node/src/integrations/tracing/clickhouse/types.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| import type { Span } from '@opentelemetry/api'; | ||
| import type { InstrumentationConfig } from '@opentelemetry/instrumentation'; | ||
|
|
||
| export interface ClickHouseInstrumentationConfig extends InstrumentationConfig { | ||
| /** | ||
| * Hook called before the span ends. Can be used to add custom attributes. | ||
| */ | ||
| responseHook?: (span: Span, result: unknown) => void; | ||
|
|
||
| /** | ||
| * Database name to include in spans. | ||
| */ | ||
| dbName?: string; | ||
|
|
||
| /** | ||
| * Whether to capture full SQL query text in spans. | ||
| * Defaults to true. | ||
| */ | ||
| captureQueryText?: boolean; | ||
|
|
||
| /** | ||
| * Maximum length for captured query text. Queries longer than this will be truncated. | ||
| * Defaults to 1000 characters. | ||
| */ | ||
| maxQueryLength?: number; | ||
|
|
||
| /** | ||
| * Remote hostname or IP address of the ClickHouse server. | ||
| * Example: "clickhouse.example.com" or "192.168.1.100" | ||
| */ | ||
| peerName?: string; | ||
|
|
||
| /** | ||
| * Remote port number of the ClickHouse server. | ||
| * Example: 8123 for HTTP, 9000 for native protocol | ||
| */ | ||
| peerPort?: number; | ||
|
|
||
| /** | ||
| * Whether to capture ClickHouse execution statistics from response headers. | ||
| * This includes read/written rows, bytes, elapsed time, etc. | ||
| * Defaults to true. | ||
| */ | ||
| captureExecutionStats?: boolean; | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.