Sidekiq::AttrPackage Guidelines

Last Updated: March 23, 2026

When Sidekiq jobs fail and exhaust their retries, the job arguments are automatically logged to Datadog for debugging and monitoring. Unlike standard Rails request logging, where parameter filtering is applied, Sidekiq does not provide the same level of built-in PII filtering for job arguments.

This creates a risk: if sensitive data (e.g., emails, names, or other PII) is passed directly into job arguments, it can be unintentionally exposed in logs.

To mitigate this, Sidekiq::AttrPackage was introduced. Sidekiq::AttrPackage is implemented within the Vets-Api repository and is the standard mechanism for safely handling PII in background jobs, https://github.com/department-of-veterans-affairs/vets-api/blob/master/lib/sidekiq/attr_package.rb .

Approach

• Sensitive attributes are stored securely in Redis.

• Only a reference key (cache key) is passed as the Sidekiq job argument.

• Jobs retrieve the actual data from Redis at runtime.

Outcome

• Prevents PII from being logged in Datadog when jobs fail.

• Keeps Sidekiq job arguments safe for logging and observability.

• Aligns background job processing with our overall data protection standards.

Key Security Boundary

❌ Without AttrPackage:
Sidekiq Args → { email: "user@example.com" } → Logged in Datadog

✅ With AttrPackage:
Sidekiq Args → { cache_key: "abc123" } → Safe to log
Redis → { email: "user@example.com" } → Never logged

Best Practices

Create once → Pass through → Delete once

Cache Key Lifecycle (Critical)

A cache_key must follow a strict lifecycle to prevent:

• PII exposure

• Redis memory leaks

• Orphaned data

Rules:

1. A cache_key is created once at the entry point (Rails app).

2. The same cache_key must be passed through all jobs/services.

3. The cache_key must be deleted exactly once, regardless of outcome.

🚫 Do NOT Create Cache Keys in Jobs or Services

Creating new cache keys inside jobs leads to:

• Orphaned PII in Redis

• Broken traceability

• Increased risk of leaks

❌ Anti-Patterns (MUST NOT DO)

1. Creating packages inside jobs

def perform(...)
  Sidekiq::AttrPackage.create(...) # ❌ NEVER
end

2. Deleting in ensure

ensure
  Sidekiq::AttrPackage.delete(cache_key) # ❌ breaks retries
end

3. Regenerating cache keys

def perform(cache_key)
  new_cache_key = Sidekiq::AttrPackage.create(cache_key) # BAD
end

4. Fallback to fetching PII outside AttrPackage

icn = attrs[:icn]
icn ||= get_icn(user_id) # ❌ weakens protection

✅ Correct

def perform(cache_key)
  data = Sidekiq::AttrPackage.find(cache_key)
end

🔁 Retries Must Reuse the Same Cache Key

• Jobs and services should never generate a new cache key

• The original cache_key must remain valid for the entire run and for retries

Why:

• Sidekiq retries reuse the same job arguments

• Creating new keys breaks consistency and leaks old data

🧹 Always Clean Up Cache Keys

A cache_key must be deleted in ALL execution paths:

✅ Required Cleanup Scenarios

• Success

• Failure (after retries exhausted)

• Manual error handling

Recommended Pattern

Success Failure Clean Up - REQUIRED

def perform(cache_key)
  data = Sidekiq::AttrPackage.find(cache_key)

  # process job...
ensure
  Sidekiq::AttrPackage.delete(cache_key)
end

Retry Cache Key Clean up - REQUIRED for retries

  sidekiq_retries_exhausted do |msg, _ex|
    cache_key = msg['args'].first
     # process retries...

    Sidekiq::AttrPackage.delete(cache_key) if cache_key
  end

⚠️ Common Pitfalls to Avoid

⏳ (Optional but Recommended) Use TTL as a Safety Net

Even with proper cleanup, Redis keys should have a TTL:

• Protects against unexpected job crashes

• Prevents long-term PII retention

• The default TTL is 7 days and should be reduced if the cache_key is not required for that duration.

Sidekiq::AttrPackage.create(data, ttl: 24.hours) # Set TTL to 24 hours

TTL is a fallback and NOT a replacement for explicit deletion.

Help and feedback

• Suggest content changes to this page.

• Submit new Platform Website content.

• Get help from the Platform Support Team in Slack.

• Submit a feature idea to the Platform.