CHALLENGES
500 Million Records, No Easy Way In
The Problems We Were Brought In to Solve
Limited Real-Time Data Visibility
Over 500 million operational records sat across multiple database tables – but getting to them was slow and fragmented. Managers had no live view into fleet, finance, or operational KPIs, leaving decisions dependent on delayed, incomplete information rather than what was actually happening on the ground.
Manual Data Querying Bottleneck
Every report needed a SQL-skilled team member to write and run it manually – up to 15 minutes per request. Non-technical staff had no independent path to operational data, creating a constant queue between teams and the numbers they needed to do their jobs.
Inefficient Business Analysis
Fuel usage, route profitability, and driver performance all required someone with advanced SQL knowledge to analyse. Operations and finance managers couldn’t run ad-hoc analysis on their own, which slowed strategic planning and made it harder to respond quickly to operational issues.
High Time and Resource Costs
Over 20 hours every week were going into manual data extraction across teams. That time cost compounded across departments – reducing agility, pulling people away from higher-value work, and slowing the pace at which the business could act on what its data was telling it.
Large Dataset Performance Issues
Queries regularly returned millions of records, putting pressure on response times and making real-time access impractical. Without automatic result-size controls in place, large queries ran slowly and inconsistently – a significant problem for a business that needed fast answers at scale.
Data Quality and Consistency Problems
Similar column names across tables caused JOIN errors. Different currencies and units across regions created calculation inconsistencies. These compounding data quality issues made it difficult to trust query results and added another layer of complexity to every analysis the team tried to run.
Ready to Work With a Team That Delivers?
We plan, design, and build products that work – on time and to your requirements.
WHAT WE BUILT
A Conversational AI Analytics Agent Built
Across NLP, SQL Generation, and
Real-Time Data Access
Natural Language Query System
Operations and finance managers can ask questions about routes, fuel, or driver performance in plain English and get structured answers back straight away – no SQL knowledge needed at any point. The conversational interface cuts the IT dependency entirely, connecting teams directly to the data they need without routing every request through a technical resource.
Automated SQL Generation with 95%+ Accuracy
Google Gemini 2.0 Flash sits at the core of the query generation layer – taking a plain English question, mapping it to the right tables and columns, and producing a precise SQL query. Over 95% first-attempt accuracy is maintained across both simple lookups and complex multi-table queries, with built-in validation checking syntax, safety, and logic before anything reaches the database.
Real-Time Analytics via Flask API
A Flask-based API connects the agent directly to the client’s MySQL database, returning results in under 2 seconds. Fuel usage, delivery performance, and revenue trends are all accessible in real time – giving teams the speed they need to make decisions based on current data rather than waiting on manually compiled reports.
Multi-Step Query Validation and Security
Every SQL query generated by the AI runs through a three-stage validation pipeline – syntax check, safety check, and logic check – using sqlglot before it touches the database. The system operates in read-only mode throughout, ensuring no unauthorised data changes can occur while keeping full analytical access open to authorised users.
Domain Intelligence and Continuous Learning
The AI was trained with 300+ lines of transport and logistics-specific context – covering transport terms, metrics, and operational workflows specific to the client’s business. Every query-response pair is logged and used to refine the model, so the agent gets progressively more accurate and context-aware as real usage builds over time.
Data Cleaning and Engineering
The build started with cleaning and structuring over 50 million records – removing duplicates, mapping table relationships, and validating joins. This data foundation was the prerequisite for reliable AI training and consistent query output across the full system.
Query Complexity Classification
A multi-level classification system analyses user intent and assigns each query a complexity level – simple, moderate, or complex – before SQL generation begins. Context is maintained across the conversation, ensuring the right type of query is generated every time regardless of how the question is phrased.
Chat Persistence and History
Chat history is stored across two layers – real-time JSON logs for immediate conversational context and a long-term database archive for historical reference. This dual-layer approach gives the agent continuity across sessions and provides the data needed for ongoing model refinement.
Intelligent Response Formatting
Raw query results are never returned as-is. Units, currency symbols, and structured table formatting are applied automatically to every response – so insights reach the team in a form they can read and act on without any additional interpretation step.
API Rate Limiting and Uptime
A smart key rotation system with exponential backoff handles traffic spikes without hitting API rate limits. 99%+ uptime was maintained even during heavy usage periods – keeping the analytics agent consistently available across all teams and departments.
Self-Learning Query Refinement
Every successful query-response pair is logged and fed back into the model refinement process. The agent becomes more accurate over time as it identifies recurring patterns in real usage – improving both SQL generation quality and contextual relevance without manual retraining.
Got a Product Idea You Want to Build?
We work with companies at every stage – from early concept to full-scale delivery. Tell us where you are and we’ll take it from there.
Process
Our AI Agent Development Approach
Six structured phases – each one building directly on the last –
from raw data preparation through to a self-improving analytics agent in production.
Phase 01
Data Cleaning and Engineering
The build started with over 50 million records spread across multiple tables. We removed duplicates, mapped table relationships, validated joins, and structured the data foundation the entire AI layer would depend on for accurate training and reliable query output.
Phase 02
Business Context & Domain Intelligence
We trained the AI with 300+ lines of transport and logistics-specific rules covering fuel usage, routes, driver performance, and invoicing. This domain layer ensures the agent understands industry terms and returns business-ready responses rather than generic data output.
Phase 03
Query Complexity Classification
We built a multi-level classification system that reads user intent and assigns each query a complexity level – simple, moderate, or complex – before SQL generation begins. Conversational context is maintained across sessions to keep query accuracy consistent.
Phase 04
AI-Based SQL Query Generation
Google Gemini 2.0 Flash converts each natural language question into a precise SQL query – mapping user input to the correct tables and columns, validating syntax and logic, and retrieving results in under two seconds. Non-technical users get powerful analytics through plain English alone.
Phase 05
Agent Analysis & Response Formatting
Retrieved results are processed and formatted before they reach the user – with units, currency symbols, and structured table layouts applied automatically. The agent presents insights in a clean, readable form that operations teams can act on directly.
Phase 06
Intelligent Learning & Continuous Improvement
Every query-response pair is logged and used to refine the model over time. The agent identifies recurring patterns, improves query generation accuracy, and becomes more context-aware with each interaction – getting smarter through real usage without manual retraining.
TOOLS & TECHNOLOGIES
The Stack Behind TransIQ Logistics
A purpose-built AI and data engineering stack designed to handle large-scale logistics data,
natural language processing, and real-time SQL query generation at enterprise scale.