mtrail's TrainVision cuts rail data collection time by close to 100%

For 15 years, mtrail has supported railway and infrastructure operators with deep domain expertise and technical solutions that optimize operational processes. With TrainVision, mtrail has turned that expertise into an intelligent object recognition system, built on Ultralytics YOLO, that automates and optimizes manual processes, enabling efficient anomaly detection and secure recording and validation of train information.

The challenge: Why formation data matters#

Accurate formation and traction data (the exact sequence of wagons and locomotives that make up a train, along with train length, weight, and traction details) is essential for reliable scheduling and customer guidance. Railway operators are required to report this data to central train information systems.

For passenger trains, formation data can usually be derived automatically from planning data. But for construction trains and in marshalling yards, manual entry is still often necessary. That means railway staff have to manually record the European Vehicle Number (EVN) of every wagon and locomotive, typically wearing gloves, in an active track environment. This process is slow, error-prone, and creates a genuine safety risk: every second a worker spends focused on a tablet is a second of reduced awareness of the surrounding track.

TrainVision: Real-time recognition for rail-specific data#

TrainVision uses machine learning and computer vision to automate this recognition end to end, supporting manual processes today with a clear path toward full automation. For object detection, TrainVision relies on Ultralytics YOLO to accurately locate elements like EVNs and hazard signs within a frame. Once located, mtrail's own custom-trained OCR and recognition models extract the domain-specific information, reading the EVN itself, identifying hazard classifications, or brake signs.

Fig 1. Live demo of TrainVision leveraging Ultralytics YOLO, custom-trained for EVN and wagon markings. (privately captured image).

To reach the accuracy this use case demands, mtrail trains its models on real-world imagery combined with millions of synthetic images generated in-house. As the synthesis process is fully defined and traceable, the resulting images don't need to be manually labeled, a significant time saving that lets mtrail retrain models quickly whenever a new real-world edge case (weathering, obstruction, unusual lighting) is identified.

TrainVision isn't a single, one-size-fits-all model. For each target platform, mtrail builds a tailored, platform-specific model optimized for that runtime. This lets TrainVision be integrated into a range of deployment environments, from handheld mobile devices to stationary trackside camera systems, while still processing each image in just 8–30 milliseconds.

Why Ultralytics YOLO?#

For mtrail, detection speed and edge deployment were non-negotiable requirements. TrainVision needed to run reliably on mobile devices in the field, not just in a data center, and across the platform-specific models built for each target runtime.

That performance made Ultralytics YOLO the foundation for TrainVision's object detection layer. The YOLO models are trained on real-world imagery to accurately locate elements like EVNs and hazard signs across both mobile and fixed-camera deployments. mtrail's own OCR and recognition models then take over for the domain-specific work, extracting the detailed information from what YOLO has located.

Customer benefits and results#

By automating a task that previously relied entirely on manual entry, TrainVision delivers measurable improvements for railway operators:

• Up to 100% reduction in data collection time, by eliminating manual EVN entry in the field. mtrail saw a jump from around 40% to near 100% in efficiency with Ultralytics YOLO.
• At least 90% reduction in error rate, compared to manual data entry.
• Improved personnel safety by removing the distraction of manual data entry in active track areas.
• Full automation potential for fixed installations, where stationary camera systems can validate a wide range of railway-specific measuring points, including formations, hazardous goods, braking characteristics, wagon condition, and revision dates, without manual intervention.

What's next for TrainVision#

Defects that today are often recorded manually with physical stickers could instead be detected automatically and fed directly into European rail data platforms or maintenance systems via standardized interfaces such as TAF-TSI. Combining mtrail's rail domain expertise with computer vision at this level could further increase both the efficiency and the quality of defect detection, and, by extending random sampling with stationary systems, help establish more comprehensive, continuous safety monitoring across the network.

mtrail sees significant further potential in safety checks and predictive maintenance for rolling stock. Anomaly detection is currently performed manually during rolling stock transfers between owners and railway undertakings, including the random sampling inspections defined under ISO 2859 and the General Contract for the Use of Freight Wagons, which are a natural next application for TrainVision's detection capabilities.

Interested in how Vision AI can transform rail and transportation operations? Explore Ultralytics' industry solutions, discover our licensing options, or check out the GitHub repository to get started today.