Skip to content
Six Figure Jobs
Modular logo

Modular

AI Framework Engineer

  • Remote · United States
  • Posted 3d ago
  • Full-time
  • 6+ yrs experience

Salary

$120K - $180K

Listed in the posting

Pays more than 12% of 261 ai engineer listings in USD.

This job
$120k–$180k
Typical
$203k
Most pay
$169k–$240k
Apply on company site

Benefits

  • Generous PTO
  • 401(k) match
  • Retirement plan
  • Unlimited PTO
  • Health insurance
  • Home office stipend

Skills

AI
API
API Design
Cloud Computing
C++
Developer Experience
Jax
Observability
Python
PyTorch

Nice to have

Full job description

Description

  • At Modular, we are building a next generation AI platform to power modern applications and facilitate access to cutting-edge hardware
  • The MAX Framework is our developer-facing layer: it defines the APIs developers use to express models, integrate custom kernels, orchestrate execution, iterate on quality and ship systems into production
  • As an Senior AI Framework Engineer for MAX, you will own and evolve core APIs and developer experience for inference and training of AI models
  • You will work at the intersection of API design, systems engineering, and modern AI frameworks
  • Your output will be the specifications, abstractions, and reference implementations that make MAX feel coherent, powerful, and intuitive to use — while preserving performance, portability, maintainability and TCO
  • Influence the design of the API surface for MAX: namespaces, core abstractions, extension points, programming model, compatibility guarantees and developer experience
  • Design inference APIs that support real-world serving needs: model loading, distributed inference, quantization, tokenization/pipelines, configuration surfaces, batching/streaming, and deployment-oriented ergonomics
  • Design training APIs that scale from single device to distributed execution, with clear primitives for device placement, parallelism, checkpointing, and observability
  • Create a coherent programming model across Python and Mojo-adjacent surfaces: align naming, types, and conventions; avoid leaky abstractions; define the “pit of success”
  • Drive RFCs and technical specs: write and socialize proposals; gather feedback from internal model engineers and external users; iterate towards consensus
  • Partner cross-functionally with compiler/runtime, kernels, cloud/serving, and documentation/DevRel teams to ensure APIs map cleanly to underlying capabilities
  • Build reference implementations and exemplar code: golden-path examples, architecture templates, and best-practice patterns that teams can copy
  • Set quality bars for APIs: versioning policy, deprecation strategy, test strategy, and documentation requirements

Benefits

  • A variety of fantastic health benefits (health, dental, vision insurance; life insurance etc) are available
  • A 401k plan with up to 5% match
  • Free tax advice on Carta
  • Generous work-from-home stipend of $1500 to help you improve your home office
  • Unlimited paid time off and flexible work hours
  • PhD in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 4+ years of Software Engineering or related work experience
  • Bachelor’s degree in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 6+ years of Software Engineering or related work experience. OR
  • 3+ years of work experience with Programming Language such as Python, Mojo, Rust, C++, etc
  • Master’s degree in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 5+ years of Software Engineering or related work experience. OR
  • Experience with inference and training systems (model execution graphs, compilation, runtime scheduling, distributed execution, checkpointing, performance tuning)
  • Strong understanding of modern AI frameworks and their design tradeoffs (e.g., PyTorch, JAX, TensorFlow, vLLM, XLA/MLIR-adjacent ecosystems)
  • Strong written communication: you can write specs that engineers can implement without ambiguity
  • Significant experience designing and evolving developer-facing APIs (SDKs, frameworks, or platforms)
  • High engineering standards, pragmatism, and a bias towards incremental development without compromising long term design
  • Fluency in one or more systems / performance languages (C++, Rust, Go) and one or more user-facing languages (Python; familiarity with Mojo is a plus)
  • Excellent taste for API ergonomics: naming, composability, types, error handling, configurability, and clarity
  • For candidates who fall outside of the listed requirements, we nevertheless encourage you to apply, as we may have openings at lower or higher levels than the one advertised
  • Onboarding for new hires is conducted in-person in our Los Altos, CA office

Apply on company site

Opens Modular’s own application page in a new tab.

Similar six-figure roles

Browse all Ai Engineer jobs →
Expedia logo
Expedia
USAOnsitePrincipal

Define the technical learning architecture for AI-native engineering at Expedia Group: role- and level-based pathways fo...

Posted 11d ago

Get jobs like this one by email

AI Engineer jobs · USA

Only new matching jobs, nothing else. Confirm by email; unsubscribe in one click.

Explore more

Salary

$120K - $180K

Apply