Did you tick “Bot Account” by accident?

It has been quite a while since the last progress report. A ton of progress has been made since then, but I simply didn’t get around writing a new progress report. There have been 639 commits since the last progress report. This is significantly more than the last time given how long there has been since the last progress report.

Is this rap?
Some work needs to be done on the flow.
Nice bars though.

Release _builds are slower but should still be faster than the LLVM backend.

Had us in the second half.

the resulting executables are about 20% faster and for simple-raytracer faster than LLVM _{in debug mode.}

Had us in the first half.

core::simd is fully supported _{through emulation}

Had us in the first half again.

There are still things to be done though. [link] lists things I know of that still need to be done.

You see that “though”… you thinkin’… the rhythm… I’ve broken it!
Accidentally repetitive?.. or maybe I’m just… unrollin’ it!

You go be busy… with them sanitizers… and tracebacks
I’ll just be here chilin’… and puttin’ Rust on them maps

No but seriously, great work.

^{And yes, I know that was lame/cringe.}

Eh research shows otherwise. Rust eliminates defects for a very particular set of problems, but when it comes to logical correctness it isn’t better or worse than other languages.

Can you concede, at least to yourself, that you made ^ this ^ up?

By the way, what you claimed “research shows” is so ridiculous that it’s hilarious that you wrote it while being serious.

Hell, I cheekily mentioned Python and JS in particular because the former introduced type hints and the latter triggered creating TS as a saner shield.

Btw, that wrongly-constructed URL wasn’t even an external one. We literally have web frameworks that make sure non-external URLs with invalid paths are impossible to construct. In other words, attempting to construct a wrong one would be a compile error.

Hello there. Now I feel uncomfortable. Who am I to talk in the presence of experts.

To generate the LLVM code correctly you need to run build.rs if there is any

Good point.

and run proc macros which are natively compiled compiler plugins

Hmm. When I read “Given the existence of macros”, I didn’t really think of compiler plugins. If that’s what was meant, then I apologize for what looks now like an ELI5 comment.

The compilation process can be sandboxed as a whole, but if it runs arbitrary code, a malicious crate could take over the build process and falsify the LLVM output.

Given that crater not only builds crates, but also runs tests, one would hope that such things wouldn’t sneak to painter unnoticed!

I’m afraid this question doesn’t make a lot of sense. You seem to be confused about the purpose of the painter tool, or how macros work, probably both.

Neither is painter a sandbox tool, nor do macros have the ability to “run code”, arbitrary or otherwise, anywhere.

painter is just a call graph analysis tool for the crates.io ecosystem. It does the analysis based on generated LLVM IR code (which is not “runnable” as is) from all versions of all crates.

It’s security application is to reliably find what crate releases are vulnerable if a vulnerability is found in releases of crate Foo.

Note that we already have cargo-audit and advisory-db. painter’s goal is to confirm via call graph analysis that “Yes, your crate is vulnerable. This part of your crate calls this vulnerable part of crate Foo”.

No crate code is actually run by or in painter, except the code written to run the tool itself, of course ;)

As for Rust macros, they get expanded in the parsing stage after lexing. You can see what’s literally expanded with cargo expand. Macros are long gone by the time you get to code generation.

Incidentally, painter has this current limitation listed in the README:

• LTO and optimizers are disabled to prevent inlining, but many cases exist in which the invocation is lost at a bytecode level. Source analysis can improve this. Examples of cases where an invoke is likely lost:
  • Dynamic function calls (pointers, vtables, etc.)
  • Inlining

That’s real source/expanded code lost by the time we got to the generated IR code stage. For macros to “run arbitrary code” at that stage would be quite something ;)

Public Availability of “Painter” Tool

We are proud to announce the first publicly-available tool in our Security Toolkit: Painter! Painter is an open source project that creates a complete call graph across the entire crates ecosystem to reveal how crates relate to each other. When a vulnerability exists in one crate, Painter allows users to more easily assess potential or active risks to other crates.

The tool is aimed at addressing issues and determining risks when using other tools (such as Cargo Audit). This allows users to not only determine if a vulnerable dependency exists but if the attack path is realized. Painter was created by Rust Foundation Security Engineer Walter Pearce and released for public usage in July 2023.

^ This is new(ish) info.

I’d probably have just stuck with strings as well.

And this argument works as long as nothing wrong happens. Well, something wrong happened ;)

Smashing strings together is how this bug happened.

Constructing URLs reliably should have been the obvious first takeaway, was my point, instead of pretending the issue is not there. If Url::join() is somehow too confusing for some, then there are other ways to do it with simpler API, no problem.

False dichotomy much!

See this postmortem from Cloudflare as an example.

Under “What went wrong”, point 1 and 3:

1. An engineer wrote a regular expression that could easily backtrack enormously.

3. The regular expression engine being used didn’t have complexity guarantees.

And on what needed to done, point 4

4. Switching to either the re2 or Rust regex engine which both have run-time guarantees.

See! Plenty of procedural talk in that postmortem. Plenty of corporate talk too. But you have to mention that a bad backtracking regex was used. And you have to mention that using regexes with no complexity guarantees was glaringly wrong. To not have done so would have been silly. To not even come close to mentioning those things beyond the specific error in that specific regex, and you wouldn’t have be taken seriously.

research shows otherwise

Can you share that research?

It’s about not ignoring the clear underlying cause of the bug that is screaming at everyone who reads the bug description.

Include something along the lines of “We will use the URL crate and utilize its API to avoid trivial URL construction errors like this one in the future”, and I may take your postmortem seriously.

A flawless developer does not exist, and at no point did I fault any developer directly for their development work. But that doesn’t mean we should ignore something that is/was clearly and inherently wrong with the code. You would think this is all stating the obvious.

So it’s not "just don’t write buggy code in the first place!”. It’s “this code could clearly have been written in a way that would have prevented this bug from ever taking place”.

And yes, good code matters. A good language matters. A good type system matters. A good use of a good language with its type system, patterns, abstractions, ecosystem, and all it got to offer matters. This is Rust afterall. If those things don’t matter, then we might as well let the code be written in Python or JS, and fully recommit to the church of TDD.

Funny how you got successfully distracted by the procedural failure dance, where the obvious, as expected, got zero mentions. Giving software engineering lectures seems to be right up your alley.

If I was the author of that commit, or any crates.io developer, I would have wanted to be called out for not constructing URLs correctly. That’s the obvious first fault here. Not even hinting at that would have felt so cringe.