In my previous post I mentioned somewhat off-hand that I was working on a reader lib for a navigation standard known as ARINC 424. The project has come a long way, and is at the point I felt comfortable about publishing it https://crates.io and hopefully start getting some eyes on this.
However, in the open source world very few things are unique, and surely there must be some good ARINC 424 readers out there already right? Well, I'll get into that in this post as well as some challenges that I have been having while implementing this. I think then the post has a natural breakdown like:
- What is ARINC 424?
- Project Motivation
- Language Choice (🦀 rusters rejoice)
- Status (so far)
- Challenges (so far)
What is ARINC 424?
So, aside from the linked Wiki article, I can give a little more context here. ARINC 424 is just a numbered standard that used to be owned by an OLD company called Aeronautical Radio, Incorporated (hence, ARINC). They created a lot of standards over the years since their inception in 1929. Mostly this related to radio use, but as computers came into use in aviation, they started looking at data standards there too.
1980 is when ARINC 424 was first published and detailed a record format that could sequentially be loaded with high reliability. The standard details a fixed-width format that compresses navigation information into 132-character ascii lines that can be read into a database onboard a flight computer. This was huge back in the day, and detailed all of the data validation needs of those days which mostly relied on some older ground-station-based methods of navigation.
Here is what one of the navigation aid (navaid) lines could look like in today's version of the standard:
# record for a ground-station navigation aid called a localizer
SUSAD KFATK2 IRPW K2011130 ITWN IRPWN36471081W119435663E0130003470 NARFRESNO YOSEMITE INTL 261851713
When you break some of it down, you can see how this could encode a lot of useful information:
[S] - standard record
[USA] - area
[D] - section of the database; navaids
[KFAT] - airport
[K2] - ICAO region code
[IRPW] - navaid identifier
...
[N36471081] - location latitude (N 36°47'10.81")
[W119435663] - location longitude (W 119°43'56.63")
...
This is a simplified example of course, and there are many different types of records to handle and not all of them have the same formats. To make this manageable, however, ARINC standards are documented quite well and there is a PDF that can be obtained for each revision, historic or current, in order to learn more about how to create either the data as a data provider, or how to read the data as a data user.
Now, ARINC as a company is no longer with us. But the standard is so pervasive that even recently at AERO 26' - the largest general aviation conference in Europe - I asked the folks at both Garmin and Boeing-Jeppesen how things are and they both mentioned current need for supporting ARINC 424 in their products. So how does the standard get updated if the company that owned it no longer exists? Well, it got transferred to a subdivision in SAE International (another large standards org) and several organizations in aviation work together. The primary needs nowadays are kept light:
- address new developments in navigation systems/tech and incorporate the necessary data
- fix format issues or issues in incorrect documentation that can lead to implementation problems
That's pretty much the only time a new revision will be needed. But it is still updated and the latest revision is revision 23, from around 2022. A new revision is in the works now, but not much has changed in aviation so 23 is currently the only thing needed in the world atm.
There are also other formats for navigation data, like AIXM, which is out of scope for this post, but is an XML-based data format that is way more expressive for the data, and this is generally where the industry is going, but because most equipment is built with ARINC 424 as the first-class data format to load in, so must ARINC 424 stay for now.
This is a lot of info, so I hope it was presented in a good way, but if not, shout and I can update this section to improve the narrative and make it clearer.
Project motivation
The primary motivation for me as with any project is that I had a problem and went into the rabbit hole and realized there was an opportunity to offer something better. I really wanted to be able to use the FAA CIFP exports for a separate project relating to airspace management data. Now, as alluded to before, I could use AIXM for this, but I needed something that could potentially work with COTS data loaders for flight management systems. Many flight management systems unfortunately ONLY work with ARINC 424 databases. Most providers create the data using AIXM first, since it is more expressive in code, but then translate to the ARINC 424 format, but I've noticed there is a need to have a proper data structure parsed out from the lines generated from ARINC 424 and be able to do this in an FMS where typical aviation embedded hardware/software constraints apply.
My secondary motivation that the standard is just cool to learn about. The history of its evolution is fascinating and I feel like I have learned much more about aviation navigation through this than when I had to do the ground course for an instrument pilot rating. Sure, the standard is not perfect, and I'll get to some of the challenges soon, but the way things evolve tells such an interesting story on what navigation methods were popular and when they phased out, and what experimental methods were tried across all generations where FMS data was used. Fascinating stuff in my opinion.
The tertiary motivation was that my hope is by doing this someone can build awesome stuff when given a proper implementation of the standard. I say proper, not to dig at others but because of the fact that most implementations are barely 5-10 records out of many others and only a hand full of fields. I think we should have better, and I want to help bridge the gap as this standard inevitably fades from existence.
Language choice (Rust)
Language choice is a fun topic for projects in the world of software development. I chose Rust for this project, and depending on which camp you are in, you may not care, or you will think I am an idiot and a hack, or you will think I am enlightened. The reality is my choice was guided not by anything but thinking about the aviation industry, where it is heading, and what tools might fit best for a standard that inherently captures a lot of complexity in the navigation data needed to conduct flights safely.
Where aviation tech is heading
Generally I believe that with Ferrocene becoming a certified toolchain for certain ASIL categories (categories of safety critical systems in the automotive industry) and DAL-C (category of 3rd highest criticality in aviation) that the trend means more and more will look toward Rust. Rust's spec in Ferrocene is also completing some work on certification, so again all this to me was a big positive toward Rust vs Zig or even just doing it in C which also could have been fine. I am seeing the road ahead, and I may not be correct, but I will take a bet on this being where things are heading.
Typing the specification
There's a lot of mixed type information and while it could be easy to just use something like python that doesn't care about mixed dynamic typing, that's not an option given the constraints I have around working with some embedded solutions. So I wanted to look toward a more expressive typing system that gives assurances at compile time rather than runtime. Rust fit that category, and so did some other languages, but with the previous point about Ferrocene certification this was building a more confident decision. A lot of the standard fits a class of types called sum types. In C you'd do this as tagged unions and can do some pointer hacking when needed to resolve to a type as needed. This can be a bit dangerous though if you make a mistake, and that's annoying (but it's doable, so again C is a possible choice). In Rust, sum types are built into the Enum systems as variants that can be tied to an enum definition. This makes handling dispatch of resolving ascii text to a single format based on the discriminants in the data line much nicer.
The lifetime specification mechanics in the language also played heavily as a positive towards rust. It meant I could rely on the compiler to help me make sure that most of the hot path functionality is zero-copy.
What are the cons of rust?
There are many for sure, as with any language. The type system, while super expressive and helpful for most assurances, is a burden. Sometimes it can feel like you're fighting against it, but overall this isn't a deal breaker for me. It does point out some of the gaps in my knowledge of the language from time to time, and that's actually pretty nice in my opinion. It's always fun realizing there's some new trick from a forum or random x post and realizing "oh I can use that."
I will also say that library size is not idea with rust. It is much bigger than if I were to for example make this in C. That said, I do expect with aggressive size optimization this will not be a real concern in the sense that it makes the project unusable.
Compilation is another topic, and I spent a few back-and-forths wondering about how to package this to improve compilation and ergonomics of use for others. I landed ultimately on splitting the various revisions of ARINC 424 into their own features so that you can just specify which revisions of the standard you need to reference and toss the rest. Example:
# ARINC 424-18 only
arinc424-rs = { version = "0.9", features = ["rev18"] }
Another con that I see that is not really technical is that not as many experts in aviation know rust, and this can be a challenge since one important aspect of any such project is collaboration. I do think this is overridden by the future-looking prediction I have, but it is a con nonetheless that should not be overlooked and means either isolation of expertise on one extreme, or in the more likely case, burden of knowledge transfer and slower adoption.
How it's going so far
I would say it's been going great! I believe there is a good structure to this now, and while it is a pain to have to implement common changes across several features, with some being unstable even today, I think that it's been working well and it has even helped me spot some issues in the FAA's data dump. Most issues are benign, but I did catch one that could cause a crash in FMS loading software so that was cool.
Benchmarking across builds from both vanilla rustc and ferrocene is something I just glanced at for processing speed. It would be good to compare to even incomplete readers, but since the everyone loves visuals:
rev18_faa streaming — rustc vs Ferrocene
Naturally, ferrocene is slightly slower since it will not have all of the major optimizations done that typical rust builds will use, but otherwise performance doesn't really make me concerned here.
Also, as of this week I was able to publish the crate and get it into crates.io (https://crates.io/crates/arinc424-rs) and get the documentation somewhat into a decent place. That said there is a lot of work to do.
I think on the top of my mind the proper work needed is:
- add more validators for logical rules per record
- validators across multiple records that have continuations of data (necessary for validation of what is known as "Path & Terminator" logic)
- grab a database dump from major providers (e.g. Jeppesen/Keyvan) to validate
I think I need to play more with it too to see how it feels. Ultimately the most important is how it is to integrate the lib into existing codebases. Still, I like where it is at for now and being able to read just the FAA dump is a nice thing to see :)
Challenges
This project has not been all roses. There have been some major issues getting it to just this state. I wouldn't say I hate it, but to dive into those, I'd start with the standards themselves.
The standard isn't perfect
First off, the standards cost money. If you want revision 18, you will spend about 500.00 USD. Revision 23? another 600.00 USD. Granted, these prices are not too bad considering the world of standards (looking at you NMEA, ya expensive bastards). Still, it's an annoyance that makes it understandable why there exists so many partial implementations out there. High barrier of entry for many programmers.
Speaking still to the standards, I forgot to mention there is no forward or backwards compatibility across revisions. There are some reserved spaces, but compatibility, no. Each revision requires its own review, and this is why I decided ultimately to treat each revision as a separate feature with full definitions. You can't be too sure.
Another issue is how the rules of the standard are expressed. That is you have to read a wall of text sometimes and then whiteboard how the rules tie together. An example excerpt:
Enroute Airway Sequencing.
Airways changing from one level to another level will be sequenced in order as any airway in the same level. The Airway Level Code is not used to sort airways in an ARINC 424 database.
When an airway changes from Airway Level Code B to two separate airways that are coded as L and H, the point of change will carry the B in the level field.
After some time re-reading, and re-reading you get it, but yeah it's not great. That's not even taken out of context or missing info, that's literally what you get when talking about some rules about altitude coding for roads in the sky called airways. I think you get the point.
Not always standard
Most cases, standards are what they are. Then you have the variants like the FAA's own variation of revision 18. They ship data for free which I am eternally happy for, but then it comes with a Readme that details more rules that must be coded in separately. Annoying, but hey they documented so there's that. But still, given the approach I ultimately chose, I do think at one point this will warrant a new way of storing the field definitions for later generation of code because I have been handrolling this and it is a slog.
I understand the why behind such needs, but it definitely makes life more difficult when programming this library because it just means that either you make the performance-penalizing choice to abstract things further, or you roll up the sleeves and just get cracking to copy over into special variants that are treated like another revision. For reasons I chose the latter option.
Real errors take time to confirm
Another issue that was a bit interesting was that I found errors that at times made me believe I was wrong in my approach in handling fields from the spec. However, on much deeper review I was actually correct and the validators I had were correctly pointing to issues in the data source. This isn't a big deal and don't worry, these aren't errors that would cause any issues with flying, mostly minor stuff like missing references, or improper encoding of information. I suspect most flight software producers expect this to some extent and are just building logic around some assumptions that are safe to execute upon. I know these were confirmed mistakes by the way because in the next cycle when testing one data dump from the FAA, those issues were gone (though new ones appeared).
Here's what one such error encountered on the latest cycle of data looks like in the lib:
Error: Record validation error: Invalid section and subsection combination:
context: recommended navaid reference
raw: SPACP PGSNPGFQ07-Z Q 040SN PGDB0 M FA SN PG 0700 + 01600 0 S 215602211
errors: SPACP PGSNPGFQ07-Z Q 040SN PGDB0 M FA [SN ]PG 0700 [ ][ ] + 01600 0 S 215602211
I'll need to think about error presentation, but essentially the wrapped characters are what the validator looks at, but it still needs someone with domain knowledge to know what they are looking at. In this case it is that the reference for the navaid is missing the reference section and subsections. They are blank but it is meant to be coded as DN. I know this because at this point I am almost an expert at this data standard and have been in the weeds of it since last fall :P. Still, I think this also comes back to the reason I made this; it's pretty cool to see some fruits of labor!
So anyway that's about it. I hope this was a fun read. If you want to check out the lib, it's not in a super usable state (still under 1.0), but you CAN parse records with it and inspect structured representations of the data therein. There are instructions in the README if you want to test. Cheers and I hope you find your own crazy project like this.
Github: https://github.com/birdhalfbaked/arinc424-rs
Crates.io: https://crates.io/crates/arinc424-rs
