French Startup Bets on Software to Unlock Hidden GPU Performance

The Inference Speed Problem

Software engineers who rely on Claude Code have learned to budget time differently. A complex code generation request can take hours to resolve, creating a workflow problem that Anthropic acknowledges by charging premium rates for Fast Mode access. For enterprises running AI-dependent processes, those delays translate directly into productivity loss and mounting compute costs.

Kog, a Paris-based startup, entered this conversation in May with a technical demonstration that generated significant attention within developer communities. The company showcased 3,000 tokens per second per request on standard datacenter hardware, specifically AMD MI300X and Nvidia H200 units. The pitch centers on a contrarian thesis: that conventional GPUs contain untapped performance that software optimization can unlock without requiring purpose-built inference chips.

CEO Gaël Delalleau reports that the initial demonstration generated 200 qualified business inquiries. The response suggests market appetite for inference acceleration that works with existing infrastructure rather than demanding hardware replacement.

A Physics and Security Background

Delalleau's path to founding Kog diverges from the typical AI startup trajectory. His academic training at École Polytechnique focused on solid-state physics, while his professional experience centered on offensive cybersecurity. That combination now shapes the company's engineering methodology.

The physics background instills what Delalleau describes as understanding the fundamental constraints of hardware to maximize utilization. His security work, which included four appearances as a finalist at DEFCON's Capture The Flag tournament, taught reverse-engineering at the assembly and binary level to repurpose systems beyond their intended design parameters.

Applied to GPU optimization, this mindset translates into weeks or months of dedicated research per chip architecture. With an 11-person team, Kog must choose its hardware targets carefully. The company currently supports specific datacenter GPUs and has made clear that consumer laptop chips fall outside its scope.

From Demo to Production Reality

The May demonstration achieved impressive throughput, but with a critical caveat: the Laneformer 2B model used in testing contains only 2 billion parameters. Kog has since open-sourced that model, yet the company's commercial promise hinges on replicating similar acceleration with large language models that can exceed 100 billion parameters.

Customer conversations revealed an unexpected constraint. Kog initially assumed enterprises would fine-tune smaller models to achieve faster inference. Market feedback indicated otherwise. Organizations want acceleration for the large, capable models they already use, not optimized alternatives with reduced functionality.

This discovery redirected the company's engineering roadmap. Rather than perfecting acceleration for compact models, Kog now focuses on scaling its approach to production LLMs. Delalleau projects a September milestone: demonstrating 10x speed improvement on a major model. That benchmark will determine whether the startup's methodology withstands the memory and bandwidth demands of contemporary language models.

Early Customer Profiles

Software engineering tools represent Kog's primary initial use case, but the startup has also engaged design partners in adjacent verticals. One category involves platforms that generate games or applications from natural language prompts. For these services, inference latency directly affects user retention and revenue, since faster output enables more usage cycles per session.

The Kog Inference Engine aims to serve workflows where speed creates measurable business value. Enterprises that have integrated AI into time-sensitive operations and customers willing to pay Anthropic's Fast Mode premium represent the addressable market Kog targets.

Delalleau acknowledges the market remains nascent. Many potential customers still evaluate whether AI inference speed justifies dedicated optimization investment. Kog's commercial traction will depend on demonstrating clear return on implementation for early adopters.

Technical Approach and Competitive Landscape

Kog's methodology involves hardware-specific optimization at the lowest software layers. The company examines each GPU architecture's memory bandwidth, compute characteristics, and instruction sets to extract performance gains. This differs from hardware-agnostic approaches that prioritize portability.

Other European startups pursue adjacent strategies. ZML, also based in France, developed software that bypasses Nvidia's CUDA stack to enable cross-chip inference. Delalleau positions Kog closer to Stanford's Hazy Research lab, emphasizing deeper GPU-level acceleration work.

The tradeoff inherent in Kog's approach creates both differentiation and constraint. Manual, architecture-specific optimization potentially yields greater performance gains than generic solutions. It also limits how many chip types the startup can support simultaneously. Long term, Kog plans to incorporate agent-based pipelines that could automate portions of this optimization process, expanding hardware coverage without proportional headcount growth.

European Sovereignty and Funding Strategy

France's government-backed initiatives have provided early support. Scaleway, the French cloud provider, works with Kog, while Bpifrance and the French Tech 2030 program have contributed backing. As European policymakers emphasize technological sovereignty in AI infrastructure and chip development, Kog's positioning as a European optimization layer could align with policy priorities.

The startup recently closed a seed round co-led by Varsity VC, whose partner Kamel Zeroual previously co-founded a company with Delalleau. That earlier venture, Stribe, participated in TechCrunch50's 2009 cohort but operated in an unrelated sector.

Kog's Series A timeline depends on technical proof points. Delalleau expects that demonstrating 10x acceleration on a major LLM will enable customer traction data sufficient to support the next funding round. The startup must convert business leads into deployments while proving its optimization methodology scales beyond the 2 billion parameter models already demonstrated.

The GPU Utilization Debate

Kog's core argument challenges a common assumption in AI infrastructure: that GPUs are poorly suited for inference decoding compared to specialized accelerators. Delalleau contends this view overlooks evolving GPU capabilities. Recent generations ship with substantially increased memory bandwidth, a specification critical for inference workloads that the market has not fully exploited.

Cerebras demonstrated market receptivity to purpose-built inference hardware with its May IPO performance. Yet Kog's thesis suggests significant headroom remains in conventional GPU utilization before enterprises need to replace datacenter equipment. If validated, this approach could alter capital allocation decisions for organizations facing inference scaling challenges.

The coming months will test whether Kog's low-level optimization can deliver production-grade acceleration at the scale and model sizes enterprises require. Success would position software optimization as a viable alternative or complement to custom silicon. Failure would reinforce the case for specialized inference chips and raise questions about whether GPU architectural constraints ultimately limit software-based acceleration.

For now, Kog's 200 business leads represent interest rather than validation. Converting technical possibility into commercial deployment will require proving that weeks of chip-specific engineering translates into performance gains that justify implementation complexity and ongoing maintenance.