License, Provenance & Compliance Audits
Open weights are not open source in any uniform sense: Llama community licenses, Apache 2.0, and bespoke research licenses impose very different obligations on commercial use, redistribution, and scale thresholds. Add training-data provenance, EU AI Act documentation duties, and customer-specific policies, and legal teams rightly slow adoption. A license and compliance audit gives risk owners a defensible map — which models can be used where, under what terms, with what mitigations — turning a blocking review into a shippable decision.
Delivery is equal parts legal triage and engineering evidence. You inventory every model, adapter, embedding, and dataset in the customer's pipeline with version pins and source URLs, then classify license risk with plain-language summaries and escalation paths for ambiguous cases. Provenance artifacts — data manifests, training lineage, evaluation methodology — are assembled into model cards and system documentation aligned to EU AI Act transparency expectations. Technical controls like PII-leakage probes, refusal and safety evals, and audit-log completeness checks provide the evidence behind the paperwork.
The audit must also cover operational compliance, not just licenses. You verify customer-held key custody for weights at rest, supplier transparency to the second hop, time-limited provider access with logging, and air-gap readiness where required. Retention policies for prompts, completions, and traces are defined with legal, balancing incident debugging against data minimization. Where gaps exist, you produce a remediated architecture — often a sovereign deployment pattern — rather than just a findings deck.
Package audits as fixed-fee assessments with a risk register, a permitted-model catalog, and a remediation roadmap. Follow-on retainers keep the catalog current as models and regulations evolve — a natural recurring line. Acceptance is sign-off from legal and security on the catalog plus passing evidence checks: provenance complete, evals recorded, logs queryable, and key custody demonstrated live. That sign-off unblocks every other offering in this catalog.
Closed→Open Migration Programs
The strongest wedge into open weights is a migration that pays for itself: replace closed-model calls with open equivalents behind an OpenAI-compatible URL, cut inference spend several-fold, and keep quality provably intact. Buyers arrive with prototype-to-production sticker shock or sovereignty mandates but fear the rewrite and quality risk. A benchmark-driven migration program removes both objections by making the swap measurable, reversible, and boring by design.
Execution follows a strict sequence. First, shadow the existing traffic to build a representative eval set drawn from real production, not imagined benchmarks. Second, profile candidate open models — including distilled and fine-tuned variants — on that set for quality, latency percentiles, and cost per correct completion. Third, swap via routing config with canary stages, starting at single-digit traffic with automatic rollback on quality or latency breach. Because the API surface is compatible, application code changes are typically limited to endpoint configuration and minor function-calling adjustments.
Reporting is what wins the renewal. Instead of raw accuracy, you present dollars-per-correct-task with confidence intervals across baseline and challenger, broken down by intent complexity so stakeholders see where open models win outright and where routing to larger models remains. Latency distributions, refusal-rate deltas, and user-feedback comparisons accompany the financials. Every migration ships with a rollback runbook: routing reverts in under a minute, and the closed-model path remains warm through the soak period.
Sell migration as a pilot-to-rollout program: two-week assessment producing a savings map, then phased cutover per workload with gain-share or fixed-fee pricing. Acceptance is contractual — quality parity within agreed tolerance, latency SLOs met, and metered savings verified in billing — plus a forwardable memo for finance and procurement. Completed migrations expand naturally into fine-tuning, drift monitoring, and sovereign hosting: you did the swap, you own the quality story going forward.