Senior Python AI Engineer - RAG & Agentic AI Role in Columbus
Career & Market Insight
Career Positioning: This is a senior individual contributor role at the cutting edge of AI engineering — specifically the RAG and agentic AI space, which is one of the fastest-growing and highest-demand areas in software development as of 2024–2025. Building production-grade LLM systems is a rare, highly marketable skill.
Skills You'll Develop:
- Production RAG pipeline engineering (chunking, embeddings, reranking) — in massive demand at AI startups and enterprises.
- Agentic AI frameworks: LangGraph, LangChain, CrewAI, AutoGen — the dominant orchestration tools in the industry.
- Multi-cloud deployment (Azure + AWS) with Docker and Kubernetes — evergreen DevOps skills.
- AI observability, guardrails, and security — increasingly critical as AI moves to production.
Compensation Context: No salary is listed, which is typical for Toptal marketplace roles where rates are negotiated. Senior AI engineers with 8+ years and RAG expertise command $80–$180/hr on platforms like Toptal globally. Ensure you clarify rate expectations upfront.
Green Flags: Ownership of architecture decisions, exposure to cutting-edge AI tooling, and work with enterprise-scale systems signal strong resume-building value. The distributed team setup suggests an async-friendly culture with experienced remote workers.
Watch Out For: The role is scoped as an individual contributor with significant autonomous responsibility — ideal for seasoned engineers, but potentially isolating without mentorship structures. Clarify team size and collaboration cadence in interviews.
Summary
We are seeking an experienced Python Backend Developer to design, build, and deploy scalable AI-powered applications using Retrieval-Augmented Generation, large language models, and agentic AI frameworks. The role will focus on delivering a production-grade RAG system and AI chatbot that can securely integrate with enterprise data, APIs, databases, and cloud services.
General Information
The organization is developing an AI-powered platform and requires an experienced individual contributor to build its RAG architecture and conversational AI capabilities. The developer will work closely with a distributed team and should be available for several hours of overlap with US working hours.
The project involves designing AI systems that go beyond basic prompt engineering, including multi-step workflows, autonomous agents, vector search, knowledge retrieval, memory management, and tool integration. The solution must be scalable, secure, observable, and suitable for production use.
The technology environment includes Python, FastAPI, large language models, LangGraph, LangChain, vector databases, Azure AI services, AWS, Docker, Kubernetes, and microservice-based architectures.
Tasks and Deliverables
- Design the end-to-end architecture for a scalable RAG system and AI chatbot.
- Develop Python backend services, REST APIs, and microservices using FastAPI or similar frameworks.
- Build document ingestion, chunking, embedding, indexing, retrieval, and reranking pipelines.
- Implement vector search solutions using Azure AI Search or comparable vector databases.
- Design autonomous AI agents capable of planning, reasoning, tool usage, and decision-making.
- Develop multi-step AI workflows using LangGraph, LangChain, CrewAI, AutoGen, or similar orchestration frameworks.
- Integrate commercial and open-source LLMs, including Azure OpenAI, OpenAI, Anthropic Claude, Gemini, and comparable models.
- Implement conversation memory, session management, context management, and agent collaboration patterns.
- Connect AI workflows with APIs, databases, enterprise systems, and external tools.
- Develop asynchronous, high-performance services capable of handling concurrent AI workloads.
- Implement prompt management, structured outputs, guardrails, fallback logic, and model evaluation processes.
- Establish logging, monitoring, tracing, observability, security, and error-handling standards.
- Containerize and deploy AI services using Docker, Kubernetes, Azure, or AWS.
- Translate business requirements into technical designs, delivery milestones, and production-ready AI solutions.
- Collaborate with the wider team while independently owning architecture and implementation decisions.
Required Experience
- 8 or more years of professional Python backend development experience.
- Strong experience designing REST APIs, microservices, asynchronous services, and distributed backend systems.
- Hands-on experience building production-grade RAG applications.
- Strong understanding of embeddings, document chunking, semantic search, vector indexing, retrieval strategies, and reranking.
- Hands-on experience developing AI agents and multi-step LLM workflows.
- Experience with agentic AI frameworks such as LangGraph, LangChain, CrewAI, AutoGen, or comparable platforms.
- Experience integrating LLMs through OpenAI, Azure OpenAI, Anthropic Claude, Gemini, or open-source model APIs.
- Ability to design AI architecture beyond basic prompt engineering.
- Experience integrating AI applications with APIs, databases, data pipelines, and enterprise systems.
- Experience implementing security, monitoring, logging, tracing, and observability for production services.
- Experience deploying containerized applications using Docker and cloud platforms such as Azure or AWS.
- Ability to independently translate business requirements into scalable technical solutions.
- Strong communication and collaboration skills in a distributed working environment.
- Availability for several hours of overlap with US working hours.
Frequently Asked Questions
Is this role fully remote and can I work from anywhere in Asia?
What AI frameworks and tools will I be working with?
What is the minimum experience required for this position?
Is a salary or hourly rate disclosed for this role?
Will I work independently or as part of a team?
