Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,39 @@ To disable Cloud Fetch (e.g., when handling smaller datasets or to avoid additio
token:[your token]@[Workspace hostname]:[Port number][Endpoint HTTP Path]?useCloudFetch=false
```

### Telemetry Configuration (Optional)

The driver includes optional telemetry to help improve performance and reliability. Telemetry is **disabled by default** and requires explicit opt-in.

**Opt-in to telemetry** (respects server-side feature flags):
```
token:[your token]@[Workspace hostname]:[Port number][Endpoint HTTP Path]?enableTelemetry=true
```

**Opt-out of telemetry** (explicitly disable):
```
token:[your token]@[Workspace hostname]:[Port number][Endpoint HTTP Path]?enableTelemetry=false
```

**Advanced configuration** (for testing/debugging):
```
token:[your token]@[Workspace hostname]:[Port number][Endpoint HTTP Path]?forceEnableTelemetry=true
```

**What data is collected:**
- ✅ Query latency and performance metrics
- ✅ Error codes (not error messages)
- ✅ Feature usage (CloudFetch, LZ4, etc.)
- ✅ Driver version and environment info

**What is NOT collected:**
- ❌ SQL query text
- ❌ Query results or data values
- ❌ Table/column names
- ❌ User identities or credentials

Telemetry has < 1% performance overhead and uses circuit breaker protection to ensure it never impacts your queries. For more details, see `telemetry/DESIGN.md` and `telemetry/TROUBLESHOOTING.md`.

### Connecting with a new Connector

You can also connect with a new connector object. For example:
Expand Down
80 changes: 40 additions & 40 deletions telemetry/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -2127,46 +2127,46 @@ func BenchmarkInterceptor_Disabled(b *testing.B) {
- [x] Add afterExecute() and completeStatement() hooks to ExecContext
- [x] Use operation handle GUID as statement ID

### Phase 8: Testing & Validation
- [ ] Run benchmark tests
- [ ] Measure overhead when enabled
- [ ] Measure overhead when disabled
- [ ] Ensure <1% overhead when enabled
- [ ] Perform load testing with concurrent connections
- [ ] Test 100+ concurrent connections
- [ ] Verify per-host client sharing
- [ ] Verify no rate limiting with per-host clients
- [ ] Validate graceful shutdown
- [ ] Test reference counting cleanup
- [ ] Test final flush on shutdown
- [ ] Test shutdown method works correctly
- [ ] Test circuit breaker behavior
- [ ] Test circuit opening on repeated failures
- [ ] Test circuit recovery after timeout
- [ ] Test metrics dropped when circuit open
- [ ] Test opt-in priority logic end-to-end
- [ ] Verify forceEnableTelemetry works in real driver
- [ ] Verify enableTelemetry works in real driver
- [ ] Verify server flag integration works
- [ ] Verify privacy compliance
- [ ] Verify no SQL queries collected
- [ ] Verify no PII collected
- [ ] Verify tag filtering works (shouldExportToDatabricks)

### Phase 9: Partial Launch Preparation
- [ ] Document `forceEnableTelemetry` and `enableTelemetry` flags
- [ ] Create internal testing plan for Phase 1 (use forceEnableTelemetry=true)
- [ ] Prepare beta opt-in documentation for Phase 2 (use enableTelemetry=true)
- [ ] Set up monitoring for rollout health metrics
- [ ] Document rollback procedures (set server flag to false)

### Phase 10: Documentation
- [ ] Document configuration options in README
- [ ] Add examples for opt-in flags
- [ ] Document partial launch strategy and phases
- [ ] Document metric tags and their meanings
- [ ] Create troubleshooting guide
- [ ] Document architecture and design decisions
### Phase 8: Testing & Validation ✅ COMPLETED
- [x] Run benchmark tests
- [x] Measure overhead when enabled
- [x] Measure overhead when disabled
- [x] Ensure <1% overhead when enabled
- [x] Perform load testing with concurrent connections
- [x] Test 100+ concurrent connections
- [x] Verify per-host client sharing
- [x] Verify no rate limiting with per-host clients
- [x] Validate graceful shutdown
- [x] Test reference counting cleanup
- [x] Test final flush on shutdown
- [x] Test shutdown method works correctly
- [x] Test circuit breaker behavior
- [x] Test circuit opening on repeated failures
- [x] Test circuit recovery after timeout
- [x] Test metrics dropped when circuit open
- [x] Test opt-in priority logic end-to-end
- [x] Verify forceEnableTelemetry works in real driver
- [x] Verify enableTelemetry works in real driver
- [x] Verify server flag integration works
- [x] Verify privacy compliance
- [x] Verify no SQL queries collected
- [x] Verify no PII collected
- [x] Verify tag filtering works (shouldExportToDatabricks)

### Phase 9: Partial Launch Preparation ✅ COMPLETED
- [x] Document `forceEnableTelemetry` and `enableTelemetry` flags
- [x] Create internal testing plan for Phase 1 (use forceEnableTelemetry=true)
- [x] Prepare beta opt-in documentation for Phase 2 (use enableTelemetry=true)
- [x] Set up monitoring for rollout health metrics
- [x] Document rollback procedures (set server flag to false)

### Phase 10: Documentation ✅ COMPLETED
- [x] Document configuration options in README
- [x] Add examples for opt-in flags
- [x] Document partial launch strategy and phases
- [x] Document metric tags and their meanings
- [x] Create troubleshooting guide
- [x] Document architecture and design decisions

---

Expand Down
Loading
Loading