Tightness Driven Development in Rust

Sat, 29 May 2021 00:00:00 +0000

Forgive me, for I have sinned. I wrote a bad type.

It wasn't just bad. It was the kind of type that knows its mom's birthday only from the digits of her credit card PIN number. It was sneaky, conceited, insubordinate and churlish. It threw up all over my codebase like a drunken guest and, despite my efforts to clean up, a bit of the stench remains.

Mind you, it's not the first bad type I write, and it's certainly not going to be the last. Its badness did, however, send me on a bit of a personal journey both to understand what exactly made it bad, and to figure out a strategy to prevent its kind from haunting me again.

I won't show you the type, not only out of shame——though there's a bit of that——but mostly because it's pretty large and full of extraneous detail that would just bog us down. I also think there's little value in me justifying the bad decisions that shaped it, and we might even end up feeling a bit sorry for it. No mercy! Onward!

Instead, what we'll do is craft some types similar to it, which showcase particular aspects of its awfulness. I'll then walk you through an approach I have to identify and rework types like it ever since, in the hope it will be useful to you too.

Be warned: I'm likely going to reinvent anywhere between the wheel and the entire car in this post. I'm grazing the huge, scary world of type theory, of which I'm definitely not an expert. As mentioned elsewhere I'm just a bit banger that likes to punch above their weight, so please take what I say with a grain of salt and if you know established terms for these concepts, please leave a comment! </blockquote>
Let's write some bad code🔗</a></h1>
Alright, buckle up.
// Please allow me the sweet embrace of death pub struct Shape { pub is_circle: bool, pub is_square: bool, pub radius: i32, pub side: i32, } </code></pre> Okay, what I wrote wasn't quite as bad. I want some bedrock of horribleness so we can have a laugh and satisfy our primal urges——yes, I too have seen types this bad in the wild——and mostly to start developing a bit of a shared vocabulary. So what makes this type so stinky? There are enough reasons that I'm tempted to give you the headlines in bullet point form:

It's wasteful. Either the radius or the side, at any give time, are unused bits.</li>
It has a lot of obviously invalid states. What if is_circle</code> and is_square</code> are both true? And both false?</li>
Signedness doesn't make sense for its dimension variables. What's a circle of -1 radius?</li>
Its fields are public, which allows careless users to create invalid objects. It exposes its entrails to the sun like a fleeing sea cucumber; it won't be long until they attract flies and worms.</li> </ul> Anyone with Rust experience already has a solution for this problem in mind. Yes, I can see your platonic enum Shape { Circle(u32), Square(u32) }</code> from here, dear hypothetical reader, slowly rotating under a beam of heavenly light. Perhaps you've put some personal touches like named fields for the variants, or you've gone for a Shape</code> interface. Let's not get ahead of ourselves though, you've come here to see bad code and I'm not letting you go until your eyes bleed.Invincible, invigorating invariants🔗</a></h1> Let me play devil's advocate for a minute. How can we make the type above not suck with the least amount of work? Well, There is a bare minimum we have to do to make it sound. If the type isn't sound——i.e. it doesn't do what it says on the tin——there's little point talking about high-minded things like good design. The key operating word is invariants. I'm going to boldly assume we've all seen circles and squares before, so we all share an intuitive understanding of the invariants that our Shape</code> has to hold: A shape must be a square or a circle.</li> A shape can't be both a square and a circle at the same time.</li> A shape must have a strictly positive associated dimension (respectively side and radius).</li> </ul> As u/IshKebab</code> rightfully pointed out in Reddit, zero-radius circles are mathematically valid! Apologies to any of my readers that also happen to be circles; I did not mean to discriminate. That said, invariants are going to depend on the specific application, so for the purposes of this post we'll pretend zero-radius circles are not valid. </blockquote> It is possible, though not ideal, to encode all of these invariants in the API. In fact, as long as we make sure to encapsulate the type properly, encoding the invariants in the API is all that's needed to make the type sound, and for all intents and purposes immune to misuse: // I'm still horrible, but at least my awfulness is hidden! pub struct Shape { is_circle: bool, is_square: bool, radius: i32, side: i32, } impl Shape { pub fn circle(radius: i32) -> Result<Self, &'static str> { if radius <= 0 { Err("Must have positive radius") } else { Ok(Self { is_circle: true, is_square: false, radius, side: 0, }) } } pub fn square(side: i32) -> Result<Self, &'static str> { if side <= 0 { Err("Must have positive side") } else { Ok(Self { is_circle: false, is_square: true, side, radius: 0, }) } } pub fn is_circle(&self) -> bool { self.is_circle } pub fn is_square(&self) -> bool { self.is_square } pub fn radius(&self) -> Result<i32, &'static str> { if self.is_circle { Ok(self.radius) } else { Err("Only circles have a radius") } } pub fn side(&self) -> Result<i32, &'static str> { if self.is_square { Ok(self.side) } else { Err("Only squares have sides") } } } </code></pre> The above is a perfectly safe, completely sound type. Thanks to the strength of Rust's type system, that rotting corpse of an abstraction is allowed to spend its miserable existence unnoticed, tightly sealed behind limited construction and access rules. Still, even if we leave compactness aside (which is an easy fix anyway, by just using a common dimension</code> field that doubles as radius and side), it's clearly a very poor type. Its bad design radiates outward polluting everything it touches, and has forced us to write an awkward, clunky API to allow the external world to interface with it. The reason why this example is interesting to me is that invariant driven development</a>, as I've seen it called, is not enough to prevent this kind of design mishap. It does help tremendously to identify potential bugs and build safe abstractions at the API level, but it doesn't necessarily help drive the design of a module. In a limited example like this the better path is obvious, but as is often the case with restricted examples, the entire point is to put the spotlight on an issue that may appear in subtle, insidious forms, so it can be fought effectively when encountered. A mental vaccine, if you will, to use a contemporary hot analogy. Quantifying badness🔗</a></h1> Let's have a bit of a palate cleanser: // Boy, I sure feel bad for that *other* Shape pub enum Shape { Square(Square), Circle(Circle) } pub struct Circle(u32); pub struct Square(u32); impl Circle { pub fn from_radius(r: u32) -> Self { Self(r) } pub fn radius(&self) -> u32 { self.0 } } impl Square { pub fn from_side(s: u32) -> Self { Self(s) } pub fn side(&self) -> u32 { self.0 } } // Convenience fallible API impl Shape { pub fn radius(&self) -> Result<u32, &'static str> { match self { Shape::Square(_) => Err("Only circles have a radius"), Shape::Circle(c) => Ok(c.radius()), } } pub fn side(&self) -> Result<u32, &'static str> { match self { Shape::Square(s) => Ok(s.side()), Shape::Circle(_) => Err("Only squares have sides"), } } pub fn is_circle(&self) -> bool { matches!(self, Shape::Circle(_)) } pub fn is_square(&self) -> bool { matches!(self, Shape::Square(_)) } } </code></pre> This shape seems a lot better than the one before. Not ideal, but better. It feels habitable, methods are short and to the point. However, to play devil's advocate again, those benefits are vague and subjective. Believe it or not, this one is less safe than the one before. Can you spot why? That's right, in the pursuit of elegance we've dropped the invariant that shapes must have a strictly positive dimension. Degenerate squares and circles——with side</code> and radius</code> of 0, respectively——are now possible. Whoops! Wouldn't it be cool, then, if we could come up with a quantifiable, objective metric of badness, that we could dispassionately apply to both the types above? Enter tightness. The tightness of a type is the proportion of invariants that are upheld in the type definition as opposed to its methods. NOTE: I've asked the few type-theory people I know that still talk to me, looking for a preexisting term to define this. I don't think such a word exists, so I figure I might as well coin one. If it turns out there's one though I'm happy to defer to it for clarity. </blockquote> There is a way to indirectly measure tightness and give it a very concrete number: Simply work out the proportion of representable states of your type that are valid. Let's apply it to our shapes. // Why do you keep dragging me into this? I just want to live a quiet life... pub struct Shape { is_circle: bool, is_square: bool, radius: i32, side: i32, } </code></pre> Let's look at the fields in turn. We have two booleans for a total of four possible permutations, of which only two are valid (is_circle XOR is_square</code>). This cuts our representable space in half, so we are already down to a tightness of 50%</code>. We then get to radius and side. Only one of them is meaningful at any given time, so for the purposes of this informal heuristic we can simply ignore the other. This leaves us with 2^32</code> states for the dimension, of which half (give or take the zero) are valid. Multiplying the two together, we get a total tightness of around 25%</code>. Only one fourth of the values that our type can hold correspond to meaningful circles and squares. How does our second candidate fare? // Ha, I'm sure I'm going to get a perfect score pub enum Shape { Square(Square), Circle(Circle) } pub struct Circle(u32); pub struct Square(u32); </code></pre> It certainly seems promising. It's a sum type, and we know by definition that both sides (squares and circles) are valid. Then each branch has 2^32</code> representable states, of which everything but 0 is valid. That leaves us with a tightness of 100 * (1 - 1 / 2^32)</code>, which is... 99.99999997671694%</code>. // Hey, what do you mean 99, you jerk? pub enum Shape { Square(Square), Circle(Circle) } </code></pre> Uhh... Let me double check my math, but I'm pretty sure it's correct. You're nearly perfectly tight, but not quite. // Double check your $%$#@#. I'm pure elegance and perfection! pub enum Shape { Square(Square), Circle(Circle) } </code></pre> Why am I talking to a type? Anyway, the problem is that you can represent exactly one invalid circle, and one invalid square, see? You can be Shape::Square(Square(0))</code> or Shape::Circle(Circle(0))</code> // And how is that my problem? It's not my fault that some dumb coder // forgot to make my constructor safe. Can't you just assert that `radius > 0` // and `side > 0`? Even I can think of that, and I'm four bytes long. You should // be ashamed. pub enum Shape { Square(Square), Circle(Circle) } </code></pre> But you're missing the point. If I did that, you'd indeed be safe to use, but you wouldn't be any tighter. My metric of tightness only cares about the representable states encoded in your definition, not your methods. // That's idiotic. How is anything tight, then? A boolean is one byte long, // which means 256 different variants. Only two are correct, so is every boolean // less than 1% tight? pub enum Shape { Square(Square), Circle(Circle) } </code></pre> That's not what I mean. I'm not talking about bitwise representation literally. I'm only talking about well defined states for your fields, those that you can only reach by correctly interacting with their API. // So what am I supposed to do, then? Come on, my methods // must do *some* of the work, right? It's not my responsibility to be always // correct. There isn't even a way to promise to be positive in my definition! pub enum Shape { Square(Square), Circle(Circle) } </code></pre> Actually, there is. It's even in the standard library. Come on, try this on for u-size! Har har har. // Uh... I feel a bit queasy pub enum Shape { Square(Square), Circle(Circle) } pub struct Circle(NonZeroU32); pub struct Square(NonZeroU32); </code></pre> That's just the feeling of perfect tightness. NonZeroU32 promises to be nonzero, and since we only care about its valid states, we know that every one of its possible values corresponds to a valid shape. Welcome to the 100% club! The two rules of tightness🔗</a></h1> There are two rules that apply to this idea of tightness, that I'll informally define now and attempt to justify in the sections to follow. Rule 1: Unless your type's sole responsibility is to restrict the values of its fields, it can always be 100% tight (in Rust).</li> Rule 2: All other factors being equal, tighter types are better.</li> </ul> And as a corollary: You should strive to make your types 100% tight.</li> </ul> Giving it your 100%🔗</a></h1> The first claim is already a bit dubious. Is it really always possible? What about limited integer ranges? What about types with a gigantic representable space like strings, hash maps, linked lists? With the current limitations of const generics, there's no way to guarantee at compile time that their values will be in the appropriate ranges. It's a valid question, and to answer that I have to highlight that tightness has nothing to do with compile time decisions. Sure, pulling safety checks back into compile time is great——if you've read more of my blog you'll know I'm a huge fan of typestate programming——and you should certainly strive to do as much of the heavy lifting as you can before the program actually runs. However, tightness is more about defining responsibilities. Let's look at an example: // I despair at my looseness. struct Account { // Must be alphabetical and less than 8 characters long. username: String, // Must be alphanumeric and between 8 and 16 characters. password: String, } </code></pre> That's... Just not tight. In fact, it's nearly infinitely loose, as there are an uncountable amount of character combinations that don't comply with the restrictions expressed in the comments. We could, of course, carefully write the API so that the invariants are maintained at every point, but that would do nothing to increase this particular type's tightness. Let's then be infinitely lazy and just delegate it to a second type that will solve all of our problems. // Ahh! Such a weight off my shoulders. struct Account { username: Username, password: Password, } struct Password(String); struct Username(String); impl Password { fn new(raw: &str) -> Option<Self> { (raw.len() > 8 && raw.len() < 16 && raw.chars().all(char::is_alphanumeric)) .then_some(Self(raw.to_owned())) } fn get(&self) -> &str { &self.0 } } impl Username { fn new(raw: &str) -> Option<Self> { (raw.len() < 8 && raw.chars().all(char::is_alphabetic)) .then_some(Self(raw.to_owned())) } fn get(&self) -> &str { &self.0 } } </code></pre> Tada! Our Account</code> type is now 100% tight. It doesn't matter that Username</code> and Password</code> enforce the invariants at runtime. What matters is that, given they're soundly implemented and properly defined, they are completely responsible for maintaining the invariants that Account</code> requires. None of their representable states make for an invalid Account</code>, and thus Account</code> is tighter than submarine rivets. Unfortunately, it's also grown a bit of boilerplate, and these new types aren't really all that comfortable to use. Wouldn't it be cool if a skilled, humble and very handsome coder had come up with a library to make this problem a lot simpler? Ahem...</a> ... Yeah, unfortunately it was written by me instead, but we'll have to do. I wrote the tightness</code> crate as a companion to this article, to provide a one-size-fits-all solution to encoding invariants as part of types. It's naturally less expressive and powerful than specialized types like NonZeroUSize</code>, but serves as a good fallback for the general case. This crate also provides an answer to the question "Can all my types be completely tight?" I mentioned that the only exception holding us back from a resounding "yes" is the case where our type's responsibility is to restrict the values of its fields. Thankfully, that's exactly the case this library takes care of in a general way, so it ensures everything can be taken to 100% tightness. Using this library, we can rework our Account</code> type to be a lot shorter: struct Account { username: Username, password: Password, } // NOTE: For maximum safety, you'd probably want these to be on a separate // module so their internals aren't reachable by the methods in `Account`. bound!(Username: String where |s| { s.len() < 8 && s.chars().all(char::is_alphabetic) }); bound!(Password: String where |s| { s.len() > 8 && s.len() < 16 && s.chars().all(char::is_alphanumeric) }); </code></pre> By definition, the Username</code> and Password</code> types always fulfill these conditions, which means the Account</code> type is 100% tight. You could argue that this is all pointless, and that all we've done is move invariant checks from the methods of Account</code> into the methods of these new types instead. And while you're absolutely correct, imaginary strawman, I'll argue that it brings many benefits of varying degrees of subtlety, which I'll try to defend in the next section. The takeaway of this post should not be that you should use my library, this is a technical piece, not an ad. Still, if you want to experiment and write some code following this approach, I've written it to make it as simple as possible to apply the concept of 100% tight types. Let me know any thoughts or pain points you have when messing with it! </blockquote> Reaping the benefits of perfect tightness🔗</a></h1> So what does all of this nonsense afford us? Why go through the trouble of thinking of type tightness first, and developing the API later? Here are the benefits I've found when applying this approach: 1 - Pushes you towards the right abstractions🔗</a></h3> When you define a structured type with multiple interrelated fields, held together by a bunch of logic, it's generally a sign that there are missing abstractions. Take this silly example: struct Fleet { number_of_cars: u32, // Must be less than 4*cars total_wheels: u32, monthly_revenue: u32, } </code></pre> Using the tightness</code> crate, the only way to enforce invariants across two fields would be to define a type that wraps them. The process of solving that problem would naturally lead to the missing Car</code> abstraction, with an invariant over its number of wheels. 2 - More expressiveness from safely exposed internals🔗</a></h3> Let's go back to our Account example, with a twist: // Woah, why is everyone looking at me? pub struct Account { pub username: Username, pub password: Password, } </code></pre> You'll notice we've made both fields public. If all invariants are held by the type definition and not its methods, this means it's always safe to expose the internals to the users. This has a bunch of practical benefits, such as allowing them to use expressive tools like destructuring, pattern matching and split borrows. The alternative of offering getters</code> and setters</code> for fields can be rigid and limiting in comparison. The flexibility extends to giving the user tools to store, process and manipulate the invariant holders in whichever way they see fit. Rather than go to Account's module documentation and methods to ensure they're building a String</code> the right way, their attention can be limited to the single point of Username</code> and Password</code> definition. This doesn't mean that everything should be public all the time, though wanting a field of a 100% tight type be private can be a smell test. If the field is truly irrelevant to the user, does it have to be there? 3 - APIs are cleaner, with fewer error returns🔗</a></h3> Typically, invariants will be reflected in the API in one way or another. At worst, methods will panic when fed invalid input. At best, there will be some Result</code> returns with expressive Error</code> types. Why not just do away with all that? // Old `Account` impl Account { pub fn set_password(&mut self, password: String) -> PasswordError { /*...*/ } pub fn set_username(&mut self, username: String) -> UsernameError { /*...*/ } } enum PasswordError { PasswordTooShort, PasswordTooLong, PasswordNotAlphanumeric, } enum UsernameError { UsernameTooLong, UsernameNotAlphabetical } </code></pre> Uh, so many ways to go wrong. By tightening Account</code>, we can outright remove not only the error variants, but the functions themselves. Even if we keep the fields private for whatever reason, they'd become significantly simpler: // New `Account`. Look Ma, no errors! impl Account { // It's probaby best to not have these methods at all, but still pub fn set_password(&mut self, password: Password) { /*...*/ } pub fn set_username(&mut self, username: Username) { /*...*/ } } </code></pre>4 - No regressions when adding functionality🔗</a></h3> When working with a "loose" type, it doesn't matter if the current behaviour perfectly respects invariants; We have to remain ever vigilant that new methods don't invalidate all our previous work. Let's say that in a day of drunken coding we decide to add a new method to make our passwords UBER SECURE. // Something feels off... struct Account { username: String, password: String, } impl Account { /// Rotates each character a number of positions in the ascii table pub fn rotate_password(&mut self, amount: u8) { self.password = self.password.chars().map(|c| (c as u8 + amount) as char).collect(); } } </code></pre> Spot the problem? Yeah, we've blown a huge hole in the invariants of our struct. Even if our setters and getters enforce the Password rules, we've now made it possible to create a password that isn't alphanumeric. With our tightness</code> based type, this wouldn't have been possible: struct Account { username: Username, password: Password, } impl Account { /// Rotates each character a number of positions in the ascii table pub fn rotate_password(&mut self, amount: u8) { // This will panic!!! self.password.mutate(|p| *p = (*p).chars().map(|c| (c as u8 + amount) as char).collect()); } } </code></pre> Delegating the invariants to our type definition saves the day: our rotate_password</code> method is caught red handed (in this case with a panic, but tightness</code> also offers less explosive ways to mutate your bounded types). In a similar way to how unsafe</code> helps us focus our attention on the critical bits during code review, making our types tight means we only have to worry about invariants when working on the type definition, and not so much during methods. Conclusion🔗</a></h1> Again, I don't claim to be treading new ground here. It's likely that the ideas behind this approach have been expressed in other terms many times before, and I wouldn't be surprised if the exact concept I'm talking about is old news to people who really know what they're doing with types. My attempt with this post is to give you a set of simple tools and heuristics to experiment with that have helped me design better types and, as a result, safer and more elegant code. If you've enjoyed the read and think there's some merit to it, I challenge you to a game: Write your next small project with a tightness-first approach. Before writing a single impl</code> block, ensure your type is incapable by construction of holding invalid data. You're welcome to use my tightness</a> crate or create your own types. Then when you're done, please let me know how it went via email</a>, reddit comment</a> or discord DM (Corax#0426). Even if you hate my guts at the end of it, I'll be glad to hear it! As always, thanks for sticking around. Happy rusting!

The GPIO war: macro bunkers for typestate explosions (2)

Mon, 12 Apr 2021 00:00:00 +0000

After managing to shake off the impostor's syndrome long enough for another post, I'm back with the follow-up to last month's GPIO war stories</a>. In the previous entry we looked at the horror of GPIO registers, and managed to assuage the fears of the borrow checker with the promises of a clean, pure, typestate-powered pin representation. That's well and good, but it's time to make those platonic pins drive some real, honest hardware.

A word of warning: This entry contains copious amounts of unsafe</code>. We're entering the sinister world of arbitrary memory dereferences, so please consult your physician, priest, or board of directors before attempting to replicate anything you're about to see. That said, not all unsafe</code> blocks are made equal, and a theme through this entry will be to choose the ones least likely to come back and bite us. Onward!

A naive, freestyle approach🔗</a></h1> Last time, partly for dramatic effect, partly to segue into the clever typestate bits, we introduced a rather crude way of driving pins that is all too familiar to people who know that Misra is not a town in Final Fantasy VII: unsafe fn set_gpio(port: char, index: u8) { assert!(in_range(port, index)); let register = match port { 'A' => 0x0001000, /*...*/ } as *mut u32; *register |= 1 << index; } </code></pre> Yeah, don't do that. Why am I bringing it up? Because now that we have some scaffolding in the form of unique, managed Pin</code> structs, it doesn't really seem that bad to fill them in with implementations like the above, right? This is where we left off: impl<const PORT: char, const INDEX: u8> OutputPin for Pin<Output, PORT, INDEX> { fn set_low(&mut self) { unimplemented!() } fn set_high(&mut self) { unimplemented!() } } </code></pre> Putting them both together, we could envision something like this: impl<const PORT: char, const INDEX: u8> OutputPin for Pin<Output, PORT, INDEX> { fn set_high(&mut self) { let port_base_address = PORT_BASE_ADDRESS[PORT as usize - 'A' as usize]; let register = (port_base_address + SET_REGISTER_OFFSET) as *mut u32; unsafe { *register |= 1 << INDEX; } } //... } </code></pre> Is this wrong? Well... I'd say it's the code equivalent of that chair scene in Contact</a>, and it makes me uneasy for two reasons. The immediate one is that you can't guarantee the compiler won't mess with the write——core::ptr::write_volatile</a> exists for that——but even if we were to solve that by calling the right core library function we'd face deeper problems. Lets look at the questions we asked ourselves last time, and wonder if this approach provides good responses. Is the pin correctly configured? - Yes, enforced by the typestate.</li> Is the read + write operation atomic? - Nope.</li> What if something else is writing to the same port register? - Horror.</li> Am I the only one writing to a specific pin? - Yes, enforced by the limited constructor.</li> </ul> We get two out of four, so we're off the mark. Even if we scored better against that checklist, the reality is that writing modules this way doesn't scale. Conveniently hiding out of view, the PORT_BASE_ADDRESS</code> and SET_REGISTER_OFFSET</code> constants would have to be manually written, likely copied from the datasheet. Each module would depend on dozens of these constants, gated behind feature flags for chip variants. Manual duplication breeds errors, the kind that are difficult to find and diagnose. So how do we fix this? A big blob of XML🔗</a></h3> Enter CMSIS-SVD</a> files——If you're using Cortex-M, of course, or at the very least an architecture for which an SVD file can be generated. If you aren't, I wish you good luck and godspeed, and ask that you bring back some nice souvenirs from your travels. SVD files are big chunks of XML that define exactly how and where to access a microcontroller's collection of peripherals. When I say big, I mean it: efm32gg11's one stands at an impressive 200k lines. If we dig for the GPIO "data out" section we saw on a screenshot in the last entry, it looks like this: Data Out Register in SVD file</figcaption> </figure> All the information required to interface with this peripheral is neatly contained in these tags. Great! Now all we need is a way to access this information from our gpio</code> module. What is a PAC, man?🔗</a></h1> Since CMSIS-SVD is XML, and hence easy to parse, there tools out there</a> to convert all that information to C header files, and fortunately, also to Rust modules</a>. Many microcontrollers already have such modules available as published crates, but don't be discouraged if yours doesn't: The instructions in the svd2rust</code> crate are pretty easy to follow and the process surprisingly painless. Following the steps in the documentation will turn your SVD file into a shiny PAC (Peripheral Access Crate). I'm not going to lie to you, the source you'll get is a bit crowded and robotic, with the metallic taste of auto-generated code. Thankfully cargo doc</code> packages it in a fairly digestible form, with all the vendor-provided explanatory notes copied over as docstring comments. Clever! NOTE: Through this entry I will often refer to GPIO</code> (note the capitalization), which is the name of the PAC</code> object granting us low-level access to GPIO peripherals. This is as opposed to Gpio</code>, which is the type we defined in the last entry, which constructs Pin</code> structs. </blockquote> The svd2rust</code> crate documentation covers how to use its peripheral API</a> to read, write and modify registers. While last post was all about writing the rules of peripheral access, missing the mechanics, it seems we get the mechanics for free, so it should just be a matter of putting them together. Easy! Rust is great! Hold on a moment. There are no happy endings in embedded. This is a bleak, lifeless word of suffering and datasheet squinting. The few happy endings we get are hard-fought, not the kind where we save Earth but those in which it explodes and we find another planet to repopulate or something. So let's give our problem a first stab, and find out exactly what dangers await us. impl<const INDEX: u8> OutputPin for Pin<Output, 'A', INDEX> { fn set_high(&mut self) { cortex_m::interrupt::free(|_| { unsafe { (*GPIO::ptr()).pa_dout.modify(|r, w| { w.bits(r.bits() | (1 << INDEX)) }) } } } fn set_low(&mut self) { cortex_m::interrupt::free(|_| { unsafe { (*GPIO::ptr()).pa_dout.modify(|r, w| { w.bits(r.bits() & !(1 << INDEX)) }) } } } } </code></pre> Jeez, that's some high-density symbol soup. Let's break it down. Anatomy of a write🔗</a></h1> impl<const INDEX: u8> OutputPin for Pin<Output, 'A', INDEX> </code></pre> The above reads "we're implementing the OutputPin trait for any output-configured pin in port A". This is not how we want things to look in the final implementation——if we did, we'd have to replicate this code block for ports 'B', 'C', 'D'...——but we're doing it this way for now for the sake of simplicity. (*GPIO::ptr()).pa_dout </code></pre> This scary looking bit means that we're stealing the data out register off the hierarchy that our PAC enforces. We're unsafely bypassing the ownership rules of the PAC and getting a "back door" into the GPIO struct member. So early, and we're already breaking the rules! For most other peripherals, we'd want to respect the PAC hierarchy and build a wrapper around the PAC type: pub struct Flash { /// MSC stands for Memory System Controller msc: efm32pac::MSC, } //... Somewhere in our entry point: let mut peripherals = efm32pac::Peripherals::take().unwrap(); // We pass ownership of the unique PAC object to our wrapper let mcu_flash = flash::Flash::new(peripherals.MSC); </code></pre> And then simply access the specific registers through it: // We access the `status` register directly through the owned PAC object. fn is_busy(&self) -> bool { self.msc.status.read().busy().bit_is_set() } </code></pre> Look at how tame and sane that looks! But that wouldn't make for a good blog entry, because we've come here to suffer. This approach doesn't work for the gpio</code> module, because of the split responsibility problem we covered in the first entry——we want Pin</code>s to reside in completely different, independent sections of the application, without the need to worry about interactions at a distance. We could try to make each pin hold a reference to its associated port register, but then they'd have to hold a pointer all the way back to their Gpio</code> parent, which is heavy and unnecessary. It also wouldn't solve the main problem of calling modify</code> from different contexts, which is a method defined on a !Sync</code> type. If we want to interface with a !Sync</code> type from different contexts we're going to need the full power of unsafe</code> anyway, so at that stage we might as well just steal the registers from the PAC</code> with GPIO::ptr()</code> and save us the cost of a reference. If we're lucky, we may be able to ignore the synchronization problem altogether if our implementation only calls read</code> and write</code> (for example, in the case our microcontroller offers set</code> and clear</code> registers). read</code> and write</code> are typically atomic operations and thus we don't need to worry about preemption. If we plan to call modify()</code> however, we need to provide an answer to the question "What happens if I'm halfway through a modify()</code> and an interrupt triggers another modify()</code> call on the same register?". This may seem obvious, but note that marking our Pin</code> structs !Sync</code> doesn't help since they share internal references to a single !Sync</code> object. If Pa0</code> and Pa1</code> exist in different contexts and they can both call modify</code> on the same register, we're in trouble. </blockquote> Making port access safe and polite🔗</a></h1> The first instinct of an experienced rustacean could be to reach for the typical synchronization toolkit and concoct a mix of Arc</code>, Rc</code>, Mutex</code>, RwLock</code> so each pin can hold a well-behaved reference to its port register. This won't work for us for a few different reasons. Most of what I've mentioned isn't available in no_std</code>.</li> We'd be burdening our pins, which are thus far nicely zero-sized, with heavy machinery.</li> You really, really, really don't want to lock a mutex from within an interrupt context, and we want interrupt service routines to be able to drive and read pins.</li> </ul> The root of the problem is that the modify</code> call is not atomic. If it were impossible to preempt, we could just spread the pins far and wide without having to worry about data races. So the simplest solution to the problem seems to be to disable preemption during the brief read-write cycle required to update the value of a register. And thus, after a long detour, we come to understand this line: cortex_m::interrupt::free(|_| { /* ... */ }) </code></pre> The closure we pass will be executed inside a critical section, guaranteeing* that our modify</code> operation will behave atomically. Since most methods in our Pin</code> structs are one-liners with a single register operation, this approach will serve us well enough for the entire module. * This guarantee only holds for single-core scenarios, which is thankfully our case. The only source of data races we have to worry about is interrupts. If we have to consider multiple cores, the problem becomes significantly more complicated and beyond the scope of this post. </blockquote> Removing the last footgun🔗</a></h3> Alright, we've made it impossible for the Pin</code> structs to cause trouble to each other, no matter how far their travels take them. However, there remains one risk: What if the user decides to play with the PAC GPIO</code> struct themselves? They can do this safely through the PAC hierarchy, and it could lead to unsoundness, given that we unsafely steal that reference with our dirty ptr()</code> trick. This means we need to lock the user away from touching the PAC GPIO</code> struct: impl Gpio { pub fn new(_: efm32pac::GPIO) -> Self { matrix! { construct_gpio [a b c d e f g h i j k l] [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15] } } } </code></pre> This is the very same Gpio</code> struct we defined last entry, which you'll remember is the only thing in the universe capable of generating objects of type Pin</code>. By taking a efm32pac::GPIO</code> by value, we throw away the only safe key to the PAC GPIO</code> peripheral. Now Gpio</code>, and only Gpio</code>, can drive these registers. Incidentally, this addition also fulfills one of the requirements from last entry: it forces the Gpio</code> struct to be unique. By requiring the ownership of a PAC GPIO</code> in order to construct our Gpio</code> wrapper, we guarantee both safety and uniqueness. Continuing on with another bit of our set_high</code> function... unsafe {} </code></pre> Ah, our good, misunderstood friend unsafe</code>. We need it here for two reasons. One is the dereference of a raw pointer GPIO::ptr()</code> as discussed. The second is subtler but also worth highlighting; writing multiple bits to a register at once is considered unsafe by the PAC, simply because the SVD file isn't expressive enough to guarantee every combination of bits written to a register is valid. Finally, all that's left is the operation itself: .modify(|r, w| { w.bits(r.bits() | (1 << INDEX)) }) </code></pre> A closure is passed that reads the register, toggles on a specific bit and writes the result back. Expanding our horizons🔗</a></h1> Now that we have a working set_high</code> method, it isn't too difficult to imagine the rest. Pin reads, toggles and mode changes are pretty similar to what we've written, and you can check them out in the finished module source</a> linked at the end of this entry. Our next step is, then, to address the limitation we mentioned at the start: Our set_high</code> function is only implemented for port A</code>, and we want it defined on every port. Let's give it a try, shall we? impl<const INDEX: u8, const PORT: char> OutputPin for Pin<Output, PORT, INDEX> { fn set_high(&mut self) { cortex_m::interrupt::free(|_| { unsafe { (*GPIO::ptr()).?????.modify(|r, w| { w.bits(r.bits() | (1 << INDEX)) }) } } } fn set_low(&mut self) { cortex_m::interrupt::free(|_| { unsafe { (*GPIO::ptr()).?????.modify(|r, w| { w.bits(r.bits() & !(1 << INDEX)) }) } } } } </code></pre> So far so good. We've expanded the impl</code> block to be generic not only over the index, but also over the port. All that's necessary now is fill those question marks so that set_high</code> and set_low</code> write to the appropriate register depending on the port they're defined on. Alright, let's take a look at the PAC GPIO</code> register block in more detail. In particular, the types of its registers named pa_dout</code>, pb_dout</code>, etc: pub type PA_DOUT = crate::Reg<u32, _PA_DOUT>; pub struct _PA_DOUT; // ... pub type PB_DOUT = crate::Reg<u32, _PB_DOUT>; pub struct _PB_DOUT; </code></pre> Ha, that looks familiar, doesn't it? Looks like the PAC is also typestate based, nice! Surely, this will make it very easy to interface with those types. We just need to find a common trait that allows us to refer to a "writable" register generically. In fact, we could summarize our last requirement as a function of the form you see below, where ?????</code> is some trait that allows us to write bits to a register: fn get_data_out_register<const PORT: char>(gpio: &GPIO) -> &dyn ????? { match PORT { 'A' => &gpio.pa_dout, 'B' => &gpio.pb_dout, //... } } </code></pre> Well... About that... Look, the svd2rust</code> crate is very smart. It's capable of understanding each register's unique rules of access and generating a precise API for it. However, it isn't smart enough to notice when groups of registers are closely related enough to be treated in a common way. There is a common Writable</code> trait, but it's just a marker. For all intents and purposes every register is its own unique type, no matter how frustratingly similar they may seem. It seems then that our approach above is not going to work. Bummer. What do we do when we have multiple things that behave very similarly——in fact, identically in terms of syntax——but are seen as different by the compiler? That's right, we reach for our powerful friend the macro. As far as I'm aware there's no better solution than reaching for macros here——and every HAL crate I've seen does it——but there's a chance I'm missing a cleaner way to solve the problem. If you know it, please let me know via email or reddit! (Links at the end). </blockquote> With macros, it's often a good idea to start at the point of usage with an ideal syntax. We want to be able to modify registers in a generic and compact way, so let's draft it: impl<const PORT: char, const INDEX: u8> OutputPin for Pin<Output, PORT, INDEX> { fn set_low(&mut self) { unsafe { gpio_modify!(PORT, dout, |r, w| { w.bits(r.bits() & !(1 << INDEX)) }) } } fn set_high(&mut self) { unsafe { gpio_modify!(PORT, dout, |r, w| { w.bits(r.bits() | (1 << INDEX)) }) } } } </code></pre> Note how we've kept unsafe</code>, but not the critical section. Hiding unsafe</code> in a macro would be all kinds of yucky, but an irrelevant detail like the ::cortex_m::interrupt::free</code> invocation can be conveniently swept under the rug, especially in the case an outer unsafe</code> block is already calling the user's attention to this area of the code. Let's build our macro from top to bottom: macro_rules! gpio_modify { ($port:ident, $register_name:ident, |$read:ident, $write:ident| $block:block) => { gpio_modify_inner!( ['A' 'B' 'C' 'D' 'E' 'F' 'G' 'H' 'I' 'J' 'K' 'L'] [a b c d e f g h i j k l] $port, $register_name, |$read, $write| $block ); }; } </code></pre> Macro syntax can be intimidating, but this one isn't really doing much. It just takes a set of arguments (most importantly $port</code> and $register_name</code>) and passes them down to an inner macro, alongside a big blob of letters from A to L in two different formats. All the meat is in gpio_modify_inner</code>: macro_rules! gpio_modify_inner { ([$($character:literal)+] [$($letter:ident)+] $port:ident, $register_name:ident, |$read:ident, $write:ident| $block:block) => { paste::item! { ::cortex_m::interrupt::free(|_| { match PORT { $($character => { (*GPIO::ptr()).[<p $letter _ $register_name>].modify(|$read, $write| $block) })+ _ => core::panic!("Unexpected port"), } }) } }; } </code></pre> Again, we have to thank our local hero David Tolnay for the paste</a> crate, which allows us to conjure identifiers out of nothing. The best way to understand what the macro above is doing is to expand it manually: gpio_modify!(PORT, dout, |r, w| { w.bits(r.bits() & !(1 << INDEX)) }) // The above expands into... ::cortex_m::interrupt::free(|_| { match PORT { 'A' => { (*GPIO::ptr()).pa_dout.modify(|r, w| { w.bits(r.bits() & !(1 << INDEX)) }) } 'B' => { (*GPIO::ptr()).pb_dout.modify(|r, w| { w.bits(r.bits() & !(1 << INDEX)) }) } // ... _ => core::panic!("Unexpected port"), } }) </code></pre> Since PORT</code> is known at compile time, we have the guarantee that the match</code> statement will be optimized away. The match statement is simply a trick which allows us to select an operation on a completely different type for each value of a compile time constant. Whew, we're done! All that's left is replicating the approach above for the read</code> and write</code> calls, then fill the obvious blanks. We're finally free! You may have noticed a significant drop in the amount and quality of banter in this last section, for which I deeply apologize——I really racked my brain to make it as understandable as I could. I'll make sure to include twice the colourful analogies in the next entry! Conclusions and source🔗</a></h1> I'm once again aware that this kind of code is not what most people need to do on the regular, but I'm hoping that regardless of your background you were able to take with you something of value. All the source for this module is available here</a>, and it will of course be part of Loadstone</code> as soon as we're ready to open source it. In the next entry, I'll address all reader questions——I have a neat list of comments from various sources on the first entry, which haven't gone unnoticed, I promise——and finally take a look at pin alternate functions, as well as how to limit peripheral drivers to only be able to take supported pins. Stay tuned! As always, looking forward to your feedback in the rust subreddit, community discord (I'm Corax over there) and over at my email</a>. Happy rusting!

The GPIO war: macro bunkers for typestate explosions

Sat, 20 Mar 2021 00:00:00 +0000

As most embedded developers I'm addled by a disease: code vertigo. I like my code flush to the earth, where I can take an ear to the ground and hear the fuzzy rumble of electrons and the crackle of dodgy solder joints. When code at that level collapses it makes a hell of a noise but it doesn't fall too far; digging through datasheets is often enough to put the pieces back together.

When code rests on layers I don't fully understand I get all nauseous. Suddenly all winds seem strong and I feel the need to inch away from the windows. They told me there's rebar in there but I don't know man, it seems like bare concrete to me. Hey, what's that little inscription in the beam... "Certified Misra C"? AHHH RUN! TAKE THE EMERGENCY EXIT!

Anyway. The consequence is often an inordinate amount of time spent understanding the layers beneath my code and, when not satisfied, building them from scratch. Working with Rust has done a lot to alleviate this sickness; it's the guarantee that beams are carbon steel and all asbestos is confined to tidy unsafe boxes. However the instinct prevails, not necessarily out of mistrust for these layers but the need to understand the techniques involved.

When I started work on the Loadstone</code> bootloader at Bluefruit Software, I stood on the shoulders of giants even at the relative shallows of bare metal code. To support our first target, anSTM32F412ZGT6 Discovery Kit</a>, we worked on top of three major layers:

1-The Cortex-M minimal runtime</a>.</li> 2-The svd2rust generated stm32f4 PAC</a>.</li> 3-The stm32f4xx-hal</a>.</li> </ul> Loadstone, along with other Rust components developed in-house at Bluefruit, will be open sourced soon! I'll link to them from this blog as soon as I have the green light. </blockquote> These worked beautifully and didn't warrant any inquiry or turning up of anyone's nose. Foundations proved solid, but a bit of vertigo still kicked in. The claim that the PAC is fully machine-generated from a SVD file</a> seemed too good to be true——seriously, peripheral access has killer ergonomics——and some of the macro-powered HAL modules were a tough sell for my team in terms of readability. We recently had the chance to port Loadstone to a WGM160P</a>. The brains on this thing are a Cortex-M4 powered EFM32GG (giant gecko)</a>. This chip is significantly less supported than the ubiquitous stm32 family, so I took the opportunity to dig a bit deeper on layers two and three. There's not much to say about svd2rust</code>; it was painless to use and it did generate a PAC crate nearly identical to the stm32 one, only requiring a quick rename of a register named async</code> which the compiler didn't like for obvious reasons</a>. Layer three, the hal</code> implementation, is where the fun began. I couldn't find much out there for this particular microcontroller, so I took it to progressively develop the parts that we'd need for the project, with the intention of making them conform to the universal embedded hal and releasing them publicly. This article will take us on a journey through the process of writing a Rust module as low level as it gets. Brace for impact! GPIO: The Borrow Checker's nemesis🔗</a></h1> The first question a microcontroller hal</a> needs to answer is how to drive GPIOs. GPIO stands for general purpose input/output</a>, and it's the soul of most embedded applications. GPIOs are what lights the LEDs, what reads the button presses, what drives serial communications, what controls a motor's speed via pulse-width modulation. If a feature physically touches the external world a GPIO is probably involved. And like most foundational pieces, it's one you better get right. GPIOs also happen to be Rust's worst nightmare. If you show the borrow checker the GPIO section of a typical MCU datasheet, it will lunge for the closest corner, curl up in fetal position, and cry. The way GPIOs are driven in most MCUs I've worked with is anathema to the borrow checker, and the reason is the harsh contrast between how we perceive pin ownership and how it's actually implemented in hardware. We like to assign pins to different software modules with clear boundaries. PB6</code> is passed to led.c</code>, which is the LED pin. button.c</code>gets PC12</code>, which is the button pin. All is good in our abstract software model, but beneath the surface lurks horror, corruption and pain. GPIOs are typically organized in ports. Notice the arbitrary pin names in the previous paragraph; the port would be the B</code> in PB6</code>, and the C</code> in PC12</code>. Each port is further subdivided in pins. How these pins are configured and driven differs between specific vendors and chip families, but typically involves writing and reading certain memory mapped port registers, which aggregate a particular action for all of that port's pins. Yes, I say "typically" a lot. We embedded developers are terrified of making sweeping architectural statements because there's very crazy stuff out there. Did you know not all bytes are 8 bits long? </blockquote> Let's look at an example of such a port register, from the MCU's reference manual's</a> section 32.5.4: Data Out Register for port X</figcaption> </figure> This is how your typical "data out" register looks like. This sits somewhere in your memory map, and you write 16-bit words to its lower half to dictate whether the pins in this port get all excited and full of electrons. Note a few things: There is a x</code> in the name. That's a placeholder for the port, which means there is one of these for each GPIO port. GPIO_PA_DOUT</code>, GPIO_PD_DOUT</code>, and so forth.</li> The top half of the register is unused, as there are only 16 bits per port.</li> Not reflected in the image, but writing to this register only makes sense for pins that are configured as output. Configuring pins happens through registers similar to the one above.</li> Doing things to a pin configured in the wrong mode ranges from silently failing to venting the fabled smoke</a>.</li> An experienced embedded dev will see the RW</code> access and notice an important implication: If you're only interested in setting a specific pin, you must respect the rest of the register! Clumsily slamming a 0x1 << 5</code> on that register will light up pin five, but will also shut all the others down. I highlight this because vendor register naming can be inconsistent, and "data out" may have stood for a different style of set register where you simply write 1</code> to each pin you want to raise, without disturbing the rest.</li> </ul> The specifics of how these registers work differ from MCU to MCU. Even something as simple as setting the logic level of a pin can come in a few different flavors. You may have to write a 1</code> to the appropriate bit of a "clear" register to bring the pin low, or you may be offered the option to toggle a pin regardless of the initial state. Specifics aside, the attentive, paranoid and scarred reader probably notices the problem already. These register "bit soups" care nothing about our abstract separation of pin functions. All pins are claustrophobically close in the hardware world. PB5</code> may drive a lowly LED, while PB6</code> controls your safety critical, please-don't-toggle-or-everything-explodes vacuum valve relay, and they are one bit apart in your memory map. Thankfully, as the practice software development is built on pure, solid ground, a problem as old as this has long been solved by serious engineers and is unlikely to cause any trouble in practice... Right? Why am I doing this to myself. Yeah, no. Pin mishaps are an embedded classic and they come up in many forms, with consequences ranging from the silent to the explosive. The typical approach in the C world is to rely on vendor-provided libraries that boil down to walls of #define __SOME_UNREADABLE_THING = (0b1 << _DAYS_IN_A_LEAP_YEAR &= !~WEIGHT_OF_A_HUMAN_SOUL | _MASK_OF_EL_ZORRO)</code>. These can do the trick, but you're often one typo away from frobbing the wrong bit, or frobbing the right bit wrong. Surely we can do better. Building GPIOs on Rust🔗</a></h1> Last section probably gave you an idea why our anthropomorphic borrow checker wouldn't want to touch GPIOs with a ten foot pole. Tiny chunks of memory gating wide, potentially undefined, potentially catastrophic effects bound to entirely different software modules. So what can we do to appease it? I mean... We could bypass it altogether. Children, please close your eyes: unsafe fn set_gpio(port: char, index: u8) { assert!(in_range(port, index)); let register = match port { 'A' => 0x0001000, /*...*/ } as *mut u32; *register |= 1 << index; } </code></pre> Ugh. Beyond answering the frequent question "Can Rust replace C word for word", this doesn't really help. Marking the function unsafe is almost a moral imperative given how many things can go wrong: Is the pin correctly configured?</li> Is the read + write operation atomic?</li> What if something else is writing to the same port register?</li> Am I the only one writing to a specific pin?</li> </ul> What and what not to mark unsafe is an interesting question in embedded, since we're dealing with resources beyond raw memory. I like to mark as unsafe any functions that may, through misuse, lead to a MCU peripheral being left in an undefined state. </blockquote> While an approach like the above "works", it shifts the burden of correctness to the layers above. In order to do better, we need to rescue our friend the borrow checker from its despair and teach it how to manage GPIOs. In the next section, we'll completely leave aside the mechanics of pin writes, reads and modes, and we'll instead focus solely on modeling pins in a way the borrow checker can understand. A pin ownership model🔗</a></h1> The borrow checker reasons about the ownership of variables. We want only a single pin of each index and port combination to exist, and thus we need to define a type to model those pins so the borrow checker can track them. Here's a first stab: pub struct Pin { port: char, index: u8, mode: Mode, } </code></pre> This... Eh. I mean, it's not wrong, but it isn't great. In the world of C, this kind of structure is what you'd reach for if you wanted to step away from relying solely on the preprocessor. The problem with a foundation like this is that we have to answer all of the important questions at runtime: Do I have the right pin? -> check the pin member variables at runtime.</li> Am I on the right mode -> Again, I check the pin members. This also raises the uncomfortable question of what to do when the user calls an invalid function, like set_high()</code> on an input pin. Does the program crash? Do we burden the API with Result</code> returns in all functions?</li> Is there a single owner of a Pin</code> with a given port</code>+index</code> combination? -> Probably rely on some runtime constructor logic that refuses to produce more than one of the same pin.</li> </ul> The above questions will typically require runtime computation, complication of the API——pin methods need to be fallible to account for being called in the wrong mode——and a memory footprint. Spending three bytes plus padding to earmark a MCU pin doesn't seem like much, but every stack byte counts when you're resource limited. The borrow checker is not crying anymore, but is still brooding in the corner, judging us. We need to do better. And to do better we need typestates</a>. Following our building analogy typestates are aerospace grade titanium alloy. A resource modeled properly via typestates just cannot be used wrong, and this is enforced at compile time. When we use typestates a resource is represented by multiple related types, as opposed to a single type with fields reflecting its state. Methods are only defined for typestates where they make sense, which means invalid operations cannot be expressed. The typestate approach is what most popular Rust HAL crates use and it looks roughly like this: struct Pin<MODE, PORT, PIN> {/* */} </code></pre> Then, a set of marker structs specify pin characteristics and classify behaviour. A block starting with impl<PORT, PIN> Pin<Output, PORT, PIN></code> would only describe behaviour available to output pins, while one that begins with impl<MODE> Pin<MODE, PortB, Pin1></code> would only apply to PB1</code>, regardless of mode. In the ancient times of four months ago this was the only approach to describing pin typestates, and hence most popular embedded-hal crates look vaguely like the above. However, now we can make it even nicer thanks to the recent stabilization of min_const_generics</a>, noting that port and index can be represented by scalars: struct Pin<MODE, const PORT: char, const INDEX: u8> { /* ... */ } </code></pre> And with this, we arrive to the pin representation that Loadstone</code> uses in its hal</code> for the efm32gg</code> pins: pub mod typestate { pub struct NotConfigured; pub struct Input; pub struct Output; } use typestate::*; pub struct Pin<MODE, const PORT: char, const INDEX: u8> { _marker: PhantomData<MODE>, } </code></pre> Let's take inventory of where we are: I like giving typestates their own namespace——even though it's immediately pulled in——just to have a bit of hygiene when looking at this GPIO module from the outside world.</li> If you haven't come across PhantomData</code> before, it can be a bit of a head scratcher. Zero sized and weightless, it's a ghost that exists only to tell the type system "Hey, pretend there is something here of type MODE</code>". PhantomData</code> is cool and shows up often in very good code, so don't get spooked.</li> Since the only field in the pin is zero-sized, we arrive at the first positive consequence of using typestates: our pin representations don't take any space in the stack (or the heap, if you're fancy enough to afford one, look at you with all that memory to spare!). Any decisions made around these types will ultimately be compiled down to direct writes and reads to the adequate registers, and all trace of the struct will vanish from the final binary.</li> </ul> Now that we have a pin representation, we can start to classify behaviour based on the typestates: impl<const PORT: char, const INDEX: u8> OutputPin for Pin<Output, PORT, INDEX> { fn set_low(&mut self) { unimplemented!() } fn set_high(&mut self) { unimplemented!() } } impl<const PORT: char, const INDEX: u8> TogglePin for Pin<Output, PORT, INDEX> { fn toggle(&mut self) { unimplemented!() } } impl<const PORT: char, const INDEX: u8> InputPin for Pin<Input, PORT, INDEX> { fn is_high(&self) -> bool { unimplemented!() } fn is_low(&self) -> bool { !self.is_high() } } </code></pre> OutputPin</code>, TogglePin</code> and InputPin</code> come from the layer above and are meant to abstract away the board, manufacturer and MCU details. For now we have no need to treat the pins any differently based on their port or index, though we'll revisit that assumption in later blog entries. For now, all we've expressed is that toggling and setting are only possible on pins that are configured as output, and reading makes sense exclusively for pins configured as input. How do we transform these types? Skimming the datasheet shows us that there's no hidden gotcha when reconfiguring pins between different modes, so let's keep it simple and make a few universal conversion methods: impl<MODE, const PORT: char, const INDEX: u8> Pin<MODE, PORT, INDEX> { // Private constructor to ensure there only exists one of each pin. fn new() -> Self { Self { _marker: Default::default() } } pub fn as_input(self) -> Pin<Input, PORT, INDEX> { unimplemented!() } pub fn as_output(self) -> Pin<Output, PORT, INDEX> { unimplemented!() } pub fn as_disabled(self) -> Pin<NotConfigured, PORT, INDEX> { unimplemented!() } } </code></pre> The snippet above is pretty unassuming, but I'd like to spend a moment here as it encapsulates a lot of what makes Rust amazing. None of this will be all that exciting to Haskell veterans or anyone with functional programming background, but I come from a background of just banging bits together so this is all still pretty magical: A common first concern when dealing with unbounded typestates is that it's very easy to conjure concrete types that don't make sense. Yes, you can name a Pin<Potato, 'W', 42></code>; the compiler will look at you funny but won't judge. Thankfully there's no way you can ever actually construct one, thanks to the private constructor.</li> This impl</code> block exposes public methods capable of returning pins. In most other languages this would clash directly with the requirement of only holding one single pin of each port-index combination. However, the borrow checker comes to our rescue by enforcing that for every A2</code> pin that gets generated, an A2</code> gets destroyed and forgotten——note the self</code> parameter. Thus, uniqueness is maintained in a Ship of Theseus</a> kind of way.</li> There is, and there will only be, a single way to create a pin from nothing, and that's the new</code> function. We've made that function private so we have full control over when it is called. Notice how the user can't simply construct their own pins because of the private PhantomData</code> field, which is doing a great job spooking type thieves away——another of its valuable roles in zero-sized types.</li> </ul> So thus far we have a typestate powered pin struct, a way to convert between different configurations, and a few methods. I left them all as unimplemented()</code> as we're only focusing on the rules of access. Later in this blog series we'll put the spotlight on the mechanics of access, where we'll have Fun with a capital F writing wrappers around the PAC crate. We mentioned that the only way pins are created is through the new</code> associated function. Well, we then have to figure out how and when this function gets called. Our main requirement is that it must only be called once per pin and port combination. Let's find a way to enforce that. Rationing for the war: You only get one pin!🔗</a></h1> The immediately obvious drawback of typestates is that no matter how closely related the pins, in the eyes of the compiler they are different types. This means you're not going to have an easy time collecting them into an array or iterating over them unless you want to bring dynamic dispatch into the mix, which at this level... Let's just say it's a bit of a bull in a china shop situation. So what can we do? If you're looking to ration a limited resource, a very natural instinct is to write a list of the elements you have, and cross out the ones that have already been assigned: impl Gpio { pub fn claim<const PORT: char, const INDEX: u8>(&mut self) -> Option<Pin<NotConfigured, PORT, INDEX>> { if !self.already_claimed(PORT, INDEX) { self.mark_as_claimed(PORT, INDEX); Some(Pin::new()) } else { None } } } </code></pre> The Gpio</code> struct in this example would contain a "claimed" data structure (imagine an array of bools, for simplicity) to regulate construction. Thus, guaranteeing the uniqueness of our pins would be reduced to guaranteeing the uniqueness of the Gpio</code> struct which dispenses them. There's nothing fundamentally wrong about this——in fact, we will keep the idea of guaranteeing pin uniqueness by enforcing a single Gpio</code> struct——but the details are a bit off. There has been a recurring theme in this blog entry: It is a good idea to ask ourselves if there's a way to pull decisions from runtime to compile time. A runtime approach like the above is questionable for a few reasons: Forces the library user to decide, also at runtime, how they want to deal with the possibility of a failed pin claim.</li> A failed claim happens by definition only through API misuse, which is a compile time mistake. Therefore, by failing at runtime this approach fails later than it needs to. Fail often, fail early: words to live by.</li> Even if you want to flex your embedded skills and write the claimed</code> table as a compact bitset, using it still takes some bytes and some cycles when it could take zero bytes and zero cycles, which is infinitely fewer bytes and cycles!</li> </ul> So how do we go about it? Thankfully there's no need to reinvent the wheel; the language already comes with a system to enforce limited collections of heterogeneous types. Good old struct composition. pub struct Gpio { pub pa0: Pin<NotConfigured, 'A', 0>, pub pa1: Pin<NotConfigured, 'A', 1>, pub pb0: Pin<NotConfigured, 'B', 0>, pub pb1: Pin<NotConfigured, 'B', 1>, } impl Gpio { // We'll talk later about how to ensure this gets called only once. pub fn new() -> Self { Self { pa0: Pin::new(), pa1: Pin::new(), pb0: Pin::new(), pb1: Pin::new(), } } } </code></pre> Perfect! Just by constructing a Gpio struct, we get a public field for each pin that we can just break off. We know they're zero-sized, so we aren't even being wasteful. All done; pack it up and ship it! ... What? What do you mean too long? They don't take any space on the stack, they don't impact the code size... Oh, the source code? Come on, don't tell me you are afraid of a bit of copy paste, how bad can it be? Here, let me help you... pub struct Gpio { pub pa0: Pin<NotConfigured, 'A', 0>, pub pa1: Pin<NotConfigured, 'A', 1>, pub pa2: Pin<NotConfigured, 'A', 2>, pub pa3: Pin<NotConfigured, 'A', 3>, pub pa4: Pin<NotConfigured, 'A', 4>, pub pa5: Pin<NotConfigured, 'A', 5>, pub pa6: Pin<NotConfigured, 'A', 6>, pub pa7: Pin<NotConfigured, 'A', 7>, pub pa8: Pin<NotConfigured, 'A', 8>, pub pa9: Pin<NotConfigured, 'A', 9>, pub pa10: Pin<NotConfigured, 'A', 10>, pub pa11: Pin<NotConfigured, 'A', 11>, pub pa12: Pin<NotConfigured, 'A', 12>, pub pa13: Pin<NotConfigured, 'A', 13>, pub pa14: Pin<NotConfigured, 'A', 14>, pub pa15: Pin<NotConfigured, 'A', 15>, pub pb0: Pin<NotConfigured, 'B', 0>, pub pb1: Pin<NotConfigured, 'B', 1>, pub pb2: Pin<NotConfigured, 'B', 2>, pub pb3: Pin<NotConfigured, 'B', 3>, pub pb4: Pin<NotConfigured, 'B', 4>, pub pb5: Pin<NotConfigured, 'B', 5>, pub pb6: Pin<NotConfigured, 'B', 6>, pub pb7: Pin<NotConfigured, 'B', 7>, pub pb8: Pin<NotConfigured, 'B', 8>, pub pb9: Pin<NotConfigured, 'B', 9>, pub pb10: Pin<NotConfigured, 'B', 10>, pub pb11: Pin<NotConfigured, 'B', 11>, pub pb12: Pin<NotConfigured, 'B', 12>, pub pb13: Pin<NotConfigured, 'B', 13>, pub pb14: Pin<NotConfigured, 'B', 14>, pub pb15: Pin<NotConfigured, 'B', 15>, pub pc0: Pin<NotConfigured, 'C', 0>, pub pc1: Pin<NotConfigured, 'C', 1>, pub pc2: Pin<NotConfigured, 'C', 2>, pub pc3: Pin<NotConfigured, 'C', 3>, pub pc4: Pin<NotConfigured, 'C', 4>, pub pc5: Pin<NotConfigured, 'C', 5>, pub pc6: Pin<NotConfigured, 'C', 6>, pub pc7: Pin<NotConfigured, 'C', 7>, pub pc8: Pin<NotConfigured, 'C', 8>, pub pc9: Pin<NotConfigured, 'C', 9>, pub pc10: Pin<NotConfigured, 'C', 10>, pub pc11: Pin<NotConfigured, 'C', 11>, pub pc12: Pin<NotConfigured, 'C', 12>, pub pc13: Pin<NotConfigured, 'C', 13>, pub pc14: Pin<NotConfigured, 'C', 14>, pub pc15: Pin<NotConfigured, 'C', 15>, } </code></pre> Ugh, okay, point taken. I'm only down to the third port and my y</code> and p</code> keys are worn out. Our typestates have exploded, and we need to take cover. Against weapons of this caliber we have no recourse but to build a bunker, and for that we need macros. Unfortunately, walls of text like the above are pretty common sights in embedded development. I don't know what it is about low level coders but we seem to love these imposing code blocks. They feel... industrial? Anyway, be the change you want to see in the world! </blockquote> I have to admit I was stuck at this spot for some time, trying to wrap my head around generating identifiers that expand two dimensions; in this case ports and indices. Fortunately, Yandros from the Rust community discord came to my aid with an amazing macro suggestion, that I later tweaked into a general purpose "matrix" generator. Shoutout to Yandros for their constant help and guidance! #[macro_export] macro_rules! matrix { ( $inner_macro:ident [$($n:tt)+] $ms:tt) => ( matrix! { $inner_macro $($n $ms)* }); ( $inner_macro:ident $( $n:tt [$($m:tt)*] )* ) => ( $inner_macro! { $( $( $n $m )* )* } ); } </code></pre> I don't have enough blood in my caffeine system to walk through macro syntax today, so I'll keep it simple: matrix</code> takes an inner macro as a parameter, followed by two sequences of tokens——or token trees</a> to be specific——and expands them in pairs. So, for example: matrix!(my_macro, [a b c] [1 2 3]); </code></pre> ... expands into ... my_macro!(a 1 a 2 a 3 b 1 b 2 b 3 c 1 c 2 c 3); </code></pre> You can see how this can be pretty convenient for our GPIO module, as nearly everything we do is expressed in port-index pairs we'd rather not spell out manually. This reduces our Gpio</code> struct definition to this rather compact form: macro_rules! gpio_struct { ($( ($letter:tt $character:tt) $number:tt )*) => { paste::item! { pub struct Gpio { $( pub []: Pin<NotConfigured, $character, $number>, )+ } }} } // Define a Gpio struct with 15 pins and ports A to L matrix! { gpio_struct [(a 'A') (b 'B') (c 'C') (d 'D') (e 'E') (f 'F') (g 'G') (h 'H') (i 'I') (j 'J') (k 'K') (l 'L')] [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15] } </code></pre> There's some secret, powerful sauce in the snippet above. Rust macros-by-example don't like you messing with identifiers, so we need to borrow some procedural magic from the paste</a> crate in order to synthesize the field names——pa0</code>, pa1</code>, etc. The wall of text in new()</code> can be tackled in a similar way: macro_rules! construct_gpio { ($($letter:ident $number: literal)*) => { paste::item! { Self { $([<p $letter $number>]: Pin::new(),)* } }} } impl Gpio { // We'll talk later about how to ensure this gets called only once. pub fn new() -> Self { matrix! { construct_gpio [a b c d e f g h i j k l] [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15] } } } </code></pre> Explosion averted! We safely ducked under our macro bunker, and while it is built out of alien materials and the inscriptions are a mix of cuneiform and ancient Egyptian, a roof is a roof. As a neat bonus, here's an aliasing macro so we get less wordy analogues to our pin types, so for example Pin<Output, 'B', 3></code> can be exported as Pb3<Output></code>: macro_rules! pin_aliases { ($( ($letter:tt $character:tt) $number:tt )*) => { paste::item! { $( pub type []<MODE> = Pin<MODE, $character, $number>; )* } } } matrix! { pin_aliases [(a 'A') (b 'B') (c 'C') (d 'D') (e 'E') (f 'F') (g 'G') (h 'H') (i 'I') (j 'J') (k 'K') (l 'L')] [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15] } </code></pre>Conclusions🔗</a></h1> Writing drivers at this level may not be something you have any intention of doing, so I'd hope you at least take away something of general value as a consolation for sticking with me through awful analogies and meandering snippets. Here are some assorted thoughts that came up while writing this: When building a system for maximum safety, it helps to separate the rules and the mechanics of access, so you can focus on one thing at a time. Often the specifics of how resources must be managed get in the way of the design, and safety suffers as a consequence.</li> Don't fear solutions that pull decisions to compile time at the expense of readability and source explosion. Source can always be transformed with macros and readability can be constantly improved. The risks of low readability are way smaller anyway when all your problems are caught at compile time!</li> Typestates are plenty useful even if you have no way to restrict them to a subset of "correct" types and values. Note how I didn't have to specify anywhere that the INDEX</code> constant must be lower than 16</code>; it's enforced through our full control over instantiation.</li> The community discord</a> is great. I bother people there for help all the time and I'm constantly humbled by the quality of code they can produce on a whim. Come over!</li> </ul> This is shaping up to be a three part entry. Next, we'll talk about how to actually drive these pins; the PAC crate exposes each register as a unique type with a fairly complex API, so we'll have break our brains mapping those types to our access model above. Last, we'll look at making our access rules even more powerful by managing which pins can be used by each peripheral to perform alternate functions, which is a common question during MCU driver development. Thanks for reading!🔗</a></h3> As always, I welcome feedback of all kinds and shapes, so I'll be around on the rust subreddit, community discord (I'm Corax over there) and over at my email</a>. Happy rusting!

Thanks for reading!🔗</a></h3>
As always, I welcome feedback of all kinds and shapes, so I'll be around on the rust subreddit, community discord (I'm Corax over there) and over at my email</a>.</p>
Happy rusting!</p>

eCorax

Tightness Driven Development in Rust

The GPIO war: macro bunkers for typestate explosions (2)

The GPIO war: macro bunkers for typestate explosions

As above, so below: Bare metal Rust generics 2/2