musl’s crypt() returns the string * as an error indicator when the key is over 256 bytes long. * is also exactly what the stored field contains for every account with no password, so the “hash” matches and the login succeeds.
Does anyone know why musl does this? Are they the only C library that behaves this way?
For a C library that famously tries to adhere to POSIX as much as possible, this seems like a crazy choice. From POSIX standard:
Upon successful completion, crypt() shall return a pointer to the encoded string. The first two bytes of the returned value shall be those of the salt argument. Otherwise, it shall return a null pointer and set errno to indicate the error.
Thanks yeah just woke up, finished making my way through the single-article website and still unclear if other libcs do the same thing (the article says libxcrypt picked it up, but only after musl did it).
Some more clarity ... Multiple implementations have these sentinel values (not OpenBSD). But unfortunately only musl had a sentinel value that was also used in shadow files...
Apparently because some devs didn't check for NULL which resulted in crashes. Weird choice to fix other peoples' bad programming by deviating from the spec.
To be fair (as the sshole site mentions) it is also necessary to fake a password hash check to avoid timing side channels revealing to remote clients whether users exist, so returning a sentinel value is not a horrifically unreasonable solution to make sure even naive code does approximately the right thing.
Though I am surprised that all of the code examples do not use constant-time strcmp(). Actually, I wonder if even constant-time strncmp() would still leak information about whether the account is disabled because in that case both arguments are much shorter? Food for thought.
But yeah, the crypt(3) API is very 80s C, the man page even features a lengthy description of a long-deprecated and utterly broken hashing algorithm using DES that is even implied to be broken in the text itself (and the warning message is so old it refers to "parallel computers" as though they were a new concept). Given that the default mode is to use a broken hashing algorithm, I'm shocked anyone actually uses the libc crypt(3) -- surely it's trivial to just write your own that isn't awful...?
Constant time strcmp doesn't fix it. The problem is that the user doesn't have a password at all (the hash is *). But which hashing algorithm is used is encoded in the password. So they pick a different user's hash to not leak the information that there is no password.
Alternate solution would be to hardcode a specific algorithm, but that could also leak info in many cases, I guess.
No, I get that -- my first point was meant to allude to that but maybe it wasn't phrased particularly well. The bit about constant-time strcmp() was about comparing the hashes afterwards, which was a classic attack to extract the password hash for offline cracking (though salts probably make it untenable?).
In any case, the fact you need access to an existing hash to not select something stupid like DES is all due to crypt(3) being a wacky API and while callers could work around it by selecting more reasonable defaults it's a pretty big foot gun.
The spec only defines a single error state - ENOSYS, that is basically only for systems that don't implement crypt at all.
Error handling is underspecified. The spec doesn't give you enough well-defined behaviour, meaning everyone lists pages of error-handling extensions they have added.
Pedantic, but more "author's understanding of the spec" which isn't always correct.
> assume nonstandard behavior
I go back and forth on this one (spec vs standard behavior). It feels like at some point the most widely used impl -is- the spec, as we've seen in other places (web standards).
> Upon successful completion, crypt() shall return a pointer to the hashed password [...]. Otherwise, it shall return a null pointer and set errno to indicate the error.
To the best of my knowledge, only OpenBSD and Fedora have a crypt(3) that returns NULL. Everybody else returns a string starting with "*" on error.
For example on Debian the manpage says:
> Upon error, crypt_r, crypt_rn, and crypt_ra write an _invalid_ hash to the output field of their _data_ argument, and crypt writes an invalid hash to its static storage area. This string will be shorter than 13 characters, will begin with a ‘*’, and will not compare equal to _setting_.
This is either a bad engineering choice or an intentional bug. An empty string would have been better. Why '*' instead of, say, a tilde, @ symbol, or something that generally isn't used?
Wow, what a bug. I expect the exploit is literally as simple as trying to ssh in as a user with the password has ‘*’ in /etc/shadow and typing anything as the password.
To the extent that systems that are configured in the vulnerable manner exist in the wild, I’m surprised they weren’t exploited years ago by trivial ssh bots.
P.S. Wow, I’m reminded of just how awful traditional UNIX-y C APIs and code are.
P.P.S. The musl code in question is nuts. The API it implements is awful. Seriously, here’s the manpage:
"It starts with a bold claim: "ssh accounts that should not accept a password will accept any password between 256 and 1024 characters". But then it keeps adding qualifiers:
- sshd compiled with musl
- not using PAM
- password logins enabled
- at least one account with '*' as the password "hash"
- at least one OTHER account with a valid password
The conditions are all ANDed, miss a single one and you're fine."
This doesn't seem to be nearly as big as an issue as they're making it out to be
Disabling PAM has been "generally good security advice" for decades; I remember suggestions to not use it in the 2000s.
Password logins are often enabled, especially on multi-user and default setups.
If alpine uses the * feature, it's going to be pretty common. Gentoo has it, as does Ubuntu.
If password logins are enabled, you're almost guaranteed to have at least one user with a password (root on all my boxes has one, even if I never use it and it's random garbage).
So the cheese lines up pretty well, this affects quite a bit.
If password logins are enabled, someone almost certainly has a password.
Yeah, as the dust starts to settle a bit it seems this is the largest case. Many 'stock' Alpine images, Docker images built on those, and embedded systems seem to be the main candidates.
No, no don't let all of that distract you. We gotta hype this. This is the new SSH2SHELL dude. Have we registered the domain yet? I'm scheduled on Fox News tomorrow.
They are going to be disappointed with SSH2SHELL it definitely isn't worth the hype. Using argument injection vulnerability created by a recent pull request that has been patched isn't exciting enough for a web site.
> We are making this release as fast as possible (we learned of the problem yesterday) because we have recently observed multiple cases of rapid bug rediscovery by AI agents, sometimes within hours, and so now consider long embargo periods to be a dangerous practice that primarily benefits attackers.
I think this will likely become standard. Nevertheless, a few stormy months are ahead.
How do you break back into your digital life? You wake up in a strange place. Something happened. Everything's gone. No wallet/keys/smartphone/tablet/laptop. How would you get back into your digital life given only a laptop with access to the Internet, unless something, somewhere is passwordauthenticate=yes.
i guess if i have some sort of internet access (otherwise why would i be worrying about passwords), i'd set up a new account(s) and message those friends
PasswordAuthentication=no usually means that ssh stops directly reading /etc/passwd to authenticate, not that it stops accepting passwords completely. You'll still be able to log in with a correct password, provided KbdInteractiveAuthentication is still on, PAM is available, and the PAM policy allows for it.
There's been a natural disaster and your home is now inaccessible. With global warming going the way it has, that's looking more and more likely each day. Fires, hurricanes, earthquakes, war, tsunami, etc.
If access to a system is critical, I'm going to have some way to access the system other than SSH (out-of-band access through a BMC, serial port, or VM host) that doesn't exclusively rely on the same means of authentication.
On systems that receive regular updates, SSH can fail for many reasons other than authentication problems, after all.
> You must not be in a place with many huge natural disasters.
Or I have proper backups synced across the world on different providers. The likelihood of everything failing is not a vector I worry about. If that happens I have bigger problems to worry about.
The email at the root of my identity is tied to a country code domain in my country. There are robust legal processes for enforcing my status as the registrant and regaining control of it. Once I have access to that, I can use that to gain access to everything downstream.
Is it fast? No, but that's a feature.
But it does mean I get to lean on the existing processes and identity verification already part of the government bureaucracy and the courts as my backdoor.
Remotely: An encrypted file with a seed / master passwords to all the other data, published under a public url you can remember, but not linked anywhere and not listed.
Locally: Same, but printed out, laminated, in a safe place with other documents.
I walk to my pc and type ssh-keygen. If i can’t do that I’ve got bigger problems. “I must be able to recover remotely using a memorised credential” is a terrible requirement.
Yeah your vault has password authentication. Your password manager in other words. Your vault is where you store your private ssh keys, and or the passwords for the keys.
Another thing I would recommend is having your servers in a wireguard net and only allowing ssh from inside that virtual network. There is no need to publicly expose ssh.
Wireguard allows effectively no configuration around auth, and so it's more likely that it is configured in a secure manner.
Openssh allows: password auth with weak passwords, PAM with potentially insecure PAM modules, gssappi, configurable ciphers, etc. Wireguard has none of those.
Each of those auth methods adds more possible vulnerable code paths and accidentally insecure configurations. It's very reasonable to believe that wireguard auth, since it has far fewer moving parts and configuration knobs, is likely to have fewer ways it can be insecure.
I was always uncomfortable that it's on by default and surprised anyone uses it it. It's also more cumbersome compared to just using a key. I feel like using keys is the password manager analogue for SSH.
Is this actually effective against Alpine? The default install seems to use ! as the placeholder in /etc/shadow instead of * and trying the attack doesn't seem to work.
IMO this is where languages like Rust shine. The design patterns built into the language for handling errors (Result) - force you to handle errors / function results with sentinels, null pointers, or errnos are so much more error prone than handling a Result<T,E> using a match statement.
crypt() returning `*` is insane especially since these kinds of sentinels are all over /etc/passwd
I guess this is where the culture around rust shines. You can write crypt that returns a string in rust just fine, just like you can write crypt that returns a tagged union in C. Cultural differences and syntactic sugar change which version is idiomatic.
I suppose if people started writing lots of code that crashed on error in Rust some enterprising developer might improve the function to always return a non-error value and set it to ‘*’ in error cases.
The culture is downstream of the design IMO. In general, people write the API that is obvious. In C, returning a string is obvious, while defining a custom OptionalString type is high friction. In Rust, returning Option<String> is as easy and obvious as returning a String.
All the standard Rust libs use results. The standard C libs use errno, return 0, return -1, return EOF, return nullptr, segfault, or of course UB. Other libs dream up things like returning "*". C++ has exceptions in some areas but not others, and if you turn them off, they're segfaults. I actually like C, at least more than C++, but the error handling is the worst part.
Sure, you could write deliberately bad code in Rust by unwrapping everything everywhere, but the point is that it is a distinct choice to do so. You can’t forget to handle errors - you have to make the choice to disregard them. If you forget to check for NULL in C it won’t complain and it’s dependent upon you to remember to do so.
In the vast majority of programs, errors and non-results are irrelevant at the call site. You want to program for the happy path and not wrap every single call with error checking code or `try/catch/finally`.
h = file_open(path)
data = file_read(h)
file_close(h)
return data
is the code you want to read. This essentially is:
return file_open(path).success(h -> h, file_read(h)).success(h, data -> file_close(h); data)
A language with thoughtful error handling primitives will support such code.
This is the root of a lot of the evil. Yes, humans want to program for the happy path. It is a lot more fun. However, that bias leads to so much lack of diligence to properly deal with the unhappy path.
Syntax is not going to solve logic errors. No amount of checked exceptions or forced unwrapping of a result will help with TOCTOU or any number of similar vulnerabilities in code.
You ideally want to attack the problem at two ends:
- make the syntax noise-free so you can see what is happening
- build primitives at the syscall and stdlib level for common footguns
You would still need to deal with domain-specific logic errors, but you have at least reduced the surface area exposed to casual attacks.
It is not. Being extremely restrictive in the happy path, throwing all over the place, but have generic error handling code at the top level (e.g. HTTP entry point) makes for some pretty robust, stable, easy to read software. Albeit it typically has somewhat rough user experience.
The thing is, that's a pretty good place to start. From there on, you can handle exceptions one by one, surgically, and make things nicer.
I keep hearing how diligent it is to handle errors at the call site, always by people who end up making services that either crash for preventable reasons or bury errors. Python is actually safer than Golang, and I don't mean with type hints.
returning a non-success you either handle it or pass it up the chain, potentially with not a lot of information about what originally caused it. with an exception you can retain the full information about what happened by either rethrowing it, throwing your own exception with it embedded as the cause or declaring that you don't handle it by saying you can throw it.
Yeah, checked exceptions are weird. Almost everything has some way to fail. The exact type of exception is usually meaningless, more like there are a couple of broad categories analogous to HTTP 4xx vs 5xx, and even then you usually don't want to handle it right there.
Well, for example, if you had opened a file to put the data into, you might delete the file or restore from a backup. But even if the answer is nothing, bubbling it all the way up to main is not the wrong answer. The point isn't whether you can do something, it's whether someone can do something.
The question is _where_ the error needs to be handled. It's almost never directly at the call site. And then there's a few issues with checked exceptions and Java language things (RMI, closures, etc) which make checked exceptions cumbersome. An exception (checked or not) might be worse than sum types, but they're still a thousand times better then returning an error in-band.
> The question is _where_ the error needs to be handled. It's almost never directly at the call site.
Which is why exceptions in Java can bubble up the stack? You don't have to catch it in a particular method you can let it go to the caller of the method, and so on.
Go doesn't require you to handle or pass an error upwards. You can call something that returns err and just ignore it, compiler won't care. The "if err != nil" thing is annoying too. Rust made the right compromise with !
Better design breeds better habits. In Java, you can end up handling a DNS exception 20 layers up the stack, even if your call never actually triggers a DNS lookup, simply because some method deep in the call stack declares it as possible. The culture is to propagate or wrap exceptions rather than translate them into domain-specific errors near the source.
Rust makes handling and mapping `Result` locally more ergonomic, even though `?` makes propagation easy too. Rust, however, also makes it easy to lose the context during propagation and end up with useless `Error`.
I agree but I think was also because Java didn't have sum types (or still doesn't? years I haven't read Java) and rethrowing is super easy, while Rust kinda nudges you into the right direction (if you need to rethrow at all, think if you want to rethrow verbatim or wrap with a semantic error type).
The problem with them all always comes when the ever-steady march of abstractions, features, and edge cases results in one routine throwing the same two error classes but in different places, and how you handle them is different despite being the same wrapper. Add a thousand developers and several years of churn, and poof you get extremely complex bugs that result from errors seeming to come from one place, but actually coming from another.
Having been on the receiving end of debugging that a few times, I no longer mind the in-band error handling, although admittedly I'm inexperienced enough to be confident its better. Just for the random code I've had to read where I just need to know exactly how something I didn't write and will never read again works, and whether the error is handled and how, i love reading in-band error handling, even if I too dislike the way an error laden routine must be written. Over time I've come to appreciate elegance "in the large" as opposed to the code I see in any individual function.
I always thought the ideas was great, but type system was holding it back. If you could have expressed that your thrown exceptions are the same as some other method's (e.g. a method of a class given as a parameter), and some other classes, then it could have been a lot less annoying. Powerful, even! But Java did take its sweet time to get even type parameters.
I have written very little Java, though, so this is more of an armchair analyst position.
Amen, people saw the symptoms of "needs more syntactic sugar" and misinterpreted it into "rigorous error handling is bad".
The same way that layered code should react/wrap/translate when it comes to return types, it should do the same for thrown exceptions, and checked-exceptions helps you do that. They're both results of computation.
Unfortunately this is also an area where a Rust programmer would likely just reach for the platform libc, because you really want to be compatible with exactly how the platform is hashing passwords. Once you know about the inconsistency, you can make predictable wrappers around it, but platform legacy is gonna keep throwing these curveballs.
> because we have recently observed multiple cases of
rapid bug rediscovery by AI agents, sometimes within hours, and so now
consider long embargo periods to be a dangerous practice that primarily
benefits attackers.
Seems quite reasonable. It’s quite obvious that black hats have day jobs as White hats and have been privy to embargoes for probably decades.
Embargo should last at most to the moment fix is made and code passes test suite.
Anything longer is putting systems at danger. I can recompile the package with a patch (even if Ubuntu/RHEL takes 3 weeks to do it) myself in a pinch, and I'd like to have a chance for it before the bug gets re-discovered by malicious party
What if the test suite passes in the middle of the night? Should all administrators interrupt their sleep and get up to recompile their servers immediately?
All times are the middle of the night somewhere. Now, it's true that some times are the middle of the night to more people than others, but that's rarely a usable piece of info.
That is generally the practice in professional environments, yes. Many places have an on-call rotation, and those that don't soon discover why. An embargo wouldn't fix that particular problem anyway, because time zones exist.
Not really related to the topic, but AI-popular sentences and language structure stand out to people right away these days right? I noticed some.
"If you cannot update yet, at least disable password logins: set PasswordAuthentication no in your sshd configuration and restart sshd. Either one closes the hole." (only AI would characterize a critical SSH implementation vulnerability as a hole closing game)
An attacker who can reach your SSH port can log in without knowing any password. It takes one connection, works every time, and needs no retries. (It sounds like the writer is bragging about the exploit instead of describing it. AI does this)
The attacker needs the name of an account whose password field is * (an account with no password set, which is normal for key-only accounts), and needs the machine to have at least one other account with a $5$ or $6$ password hash. The attacker then sends a very long password (257 to 1024 bytes). One user with a password is enough: it makes every account without one reachable. (same as above made stronger with the 'statement: statement2' format)
"Users coming from a directory service through musl-nscd (nslcd, winbind, sssd), since they have no shadow entry." (a human would be more clear about what a shadow entry is and give more context, I think)
Hmm the rest of the article (the remaining 70-80%) seems quite human.
Anyway, not sure why I spent 10+ minutes on this. It just jumps at me right away.
An aside but one nice thing I realize about not using LLMs is, I guess, I am not tortured by this kind of thing, because presumably I am not seeing it all the time. I can't even imagine quite being in the mode of reading this kind of text so closely like this, really. Its a security advisory, I don't really have my critical prose-reviewer hat on.
As a grad student I read and graded a lot of student philosophy papers (well before LLMs), and I just remember the exhaustion of it. It made me appreciate how much I could otherwise just read something, get information or enjoyment out of it, and not focus on each sentence construction.
Just to say, I do believe you, I get that if you have to put up with it so much in general, it must prime you to find it everywhere. You have to read so much more now, I'm sure, so you're trained to be in that mode. It makes sense! I am just sorry its like this for you and most everyone else now. It sounds exhausting.
We have clients (regulated banks and insurers of various sizes) that send us feeds via SFTP. You might be surprised what proportion of them ignore our “we strongly recommend key based authentication” and ask for password auth instead.
When I set up a client sftp drop I had the luxury of saying "key based access only"
Then it was a surprisingly customer service intensive process to get the keys from the customers. I never did manage to automate it fully. Weird enterprise systems I have never heard of with strange key formats. ssh1 keys, email mangling them. Their internal processes requiring a password.
And way to many "I am sorry but that was the private key you sent me, That key is now compromised and you will need to remove it and generate new keys anywhere it was used"
I almost regret not allowing passwords, almost, despite the headaches key based is still much better.
At least in that case, if the account is compromised we definitely didn't leak the credentials. That we don't have them to leak is one of the advantages of key-based auth.
[of course not having the credentials to leak does not protect us from this sort of bug, it is just one factor in a multipart security stance]
Previous company: probably. It could be a dealbreaker for the client, and we bent over quite readily on a number of issues considered minor (keys are best practise, but passwords are so common it is difficult to argue that they are so bad they should be refused).
Current state (now we have been bought and are effectively more than an order of magnitude bigger): I definitely don't have opportunity to make a unilateral decision like that. Even though the larger company has a more “fair enough, feel free to go elsewhere” attitude when a small client says “if you can support × that is a dealbreaker for us” then we could previously afford to take as our stance, some of the bigger clients are definitely paying us enough, and do use passwords, that someone above me would overrule a keys-only decision.
Don't most cloud providers default to that? I know I had to force some to deploy with a key and even then PasswordAuth was often on and had to be disabled.
I went and checked this. I can admin my server with Cockpit using either the web service (using my username/password) or the desktop client (via private key) without issue even with SSH password authentication disabled. I do usually disable the web service so had to reconfigure to check.
Well, OpenSSH contributes to this by mixing hashes from the shadow file. The musl code would be correct if OpenSSH passed in the hash from the user account that is being logged-in.
They instead pass in a different user’s hash (to avoid timing attacks) but end up comparing the result from musl against the actual user account, defeating musl’s simplistic safeguard to avoid this very issue.
I’d probably still call this a musl thing. Errors should result in a NULL return even if the application crashes from there, but at the very least a return value intended to not match any actual hash in /etc/shadow should be a random string or predefined UUID or something other than *.
The potential affected space is huge.. it does not matter if most docker images ship without openssh.. it does not matter that most security conscious image builders do not enable plain passwordauth.. it does not matter that ssh port is not commonly exposed.. the amount of alpine docker images in current live use over the whole internet is such a vastly huge number that you're going to bet that even given a small percentage of those 'misusing' docker and their containers being affected is still going to result in a a huge number of affected containers.
i think it's just unbelievable there isn't a bigger push to move away from c to languages that can prevent stuff like this from happening. this and just the pure unsafety. it's 2026. it's like pure negligence at this point, the equivalent of building a bridge with matchsticks because you like using it more.
How would using a language different than C help here? At it's core this bug is a logic bug, where "*" == crypt(x), because crypt in musl started returning "*" in some cases some years ago. This could be Rust. This could be Go.
Another comment said that returning a * was to avoid returning null which would crash users of the library who didn't check for null. That seems like a C smell to me
Not really. This is like if the char* were an Option<String> but you had programs which stupidly didn't check if the Option had something and assumed it always did. As others have said, Rust would not prevent this.
`Option<String>` or `Result<String>` forces the check at the callsite. Unlike C, you cannot forget that the function can error. This absolutely helps prevent issues like this.
Now, the decision to return `"*"` instead of letting poorly written code crash on `NULL` is not good either.
I don't think it's that people forget it while writing the code, it's that people want their program to work with as little effort as possible and decide to ignore it "for now". And that definitely can happen with Option and Result as well. There's no protection for not checking return values, and it's probably not even possible to protect against (in entirety)
Well they would use `unwrap` if they were lazy, but at least that would be quite visible in the code and would not result in undefined behavior due to operating on a null value. And there might be a group of developers who first choose to use .unwrap. I hope it is the minority.
Given how easy it is to just forward that error to the caller, I think most Rust developers would just choose to do that.
While dereferencing a null pointer is technically UB, it practice it leads to a safe trap. So .unwrap is the right comparison. If you always forward the error to the caller, it will also just lead to some random error message at some point, not really different to a crash.
> There's no protection for not checking return values, and it's probably not even possible to protect against (in entirety)
That's exactly what you get with Result and Option, it's impossible to forget.
Compare:
char* hash = crypt(key, salt);
// We crash here on the hash derefence, it's no clear from the code
// that we were supposed to check for NULL or that we forgot.
if (!strcmp(hash, expected)) {
}
To:
let result: Result<Vec<u8>, Error> = crypt(key, salt);
// We MUST write this line, which is a very clear signal
// that we can panic here.
// This can be grepped for or forbidden.
let hash: String = result.unwrap();
// if result == expected { would type error
if hash == expected {
}
Sure, but it's far better than the C code and I don't think in Rust a library would start returning `Ok` with sentinel values to "fix" shit calling code. Maintainers would, rightfully, tell people to fix their code.
Don't get me wrong, I think rust has some great features and does some things extraordinarily good. But at the same time I can see that for 99% of cases it's just as prone to bugs, including security and supply chain attacks, as any other language. I think I could just as much argue that any "->" is clearly de-referencing that needs checking, which is also pretty easy to see.
this is theoretically, empirically, and objectively wrong. akin to saying that you can get into an accident in a truck as well as a motorcycle, so in 99% of cases there is no difference in safety.
different kind of programming languages change the kinds of bugs and their properties, but it does not change the amount of bugs per line of code. in respect to C and rust for example, rust may have less memory bugs - which LLMs seems to be able to spot now - but it did increase supply chain attacks for example.
> that only tries to show a single study is flawed, nothing else
no the study failed to show any difference in bug count for a specific set of common used languages, where C and Python were both part of the dataset.
I don't think programming languages have a big impact on bugs per line when every line is written by a human. Now given that I was talking about bug counts, not severity.
it showed errors in the methodology of an earlier study, it did not produce a novel result.
kind of reminds me how the "research" suggests that there are no practical differences between dynamically typed and statically typed languages...except we know from practical experience that there is, and the literature there is horribly flawed.
we have real-world evidence that rust leads to significantly less bugs in the real world
> it showed errors in the methodology of an earlier study, it did not produce a novel result.
It showed that there's no big difference. From the link:
> Combining corrections for all of these aforementioned items, the reanalysis revealed that only 4 out of the original 11 languages correlated with abnormal defect rates, and even for those the effect size is exceedingly small.
--
> we have real-world evidence that rust leads to significantly less bugs in the real world
no, we only have evidence for memory bugs, that's not what I was talking about
>because crypt in musl started returning "*" in some cases some years ago
In Rust, the type of the return value would be a `Result<T, E>`, which means the chain of events that led to this would never have started in the first place.
In C the return value would be NULL, except it was changed by musl at some point.
Edit: at the beginning, crypt always returned a value. So hypothetical Rust return type would be a String, and then it would be retained for backward compatibility. Or something else would happen, this is a complex logic bug triggered by a 25 year old chain of events.
That's what led some people to write code where it crashes without checks. Which is still their mistake. But if the type was something that can't be implicitly used like an always-present value, that would have prevented the situation, even with those same people.
By "can't be used as a regular value", I meant that it would be a compile-time error. That you have to explicitly opt into skipping the check, e.g. with `unwrap` in Rust, or with something like `!` in languages where `string` and `string?` are different types (like TypeScript, Kotlin, Swift, C#, etc.)
Trying to change a function return type from e.g. `string` to `string?` would make the backwards-incompatible breaking change immediately visible, and break the builds of all code that relied on not having to check it back when it was `string` - at compile time.
I write firmware where the program has to run for years unattended.
You are right, C with manual memory management, no lifetime management, lack of phat pointers and a culture of leet has problems with deranged maddening bugs that other languages don't have.
But this bug first shows how inband signaling is bad news usually. But worse it's a failure to adhere to the spec properly. If you implement a library and you don't adhere to the spec you're going to bone someone.
Perhaps even worse is obviously the spec doesn't come with any standard testing framework. That was also the root of Apples infamous goto bug.
Rust would not save you here as this is a logic bug. Nor would, well, really any other language. I suppose someone might claim that Rust's result type would save you but it really wouldn't: I can easily make the error type a String or pretty much any other type with little consequence.
Rust would absolutely have saved you here. Even with `Result<String, String>`, which is pretty poor design, there's no confusing `Ok(s)` with `Err(s)`. Sure you can go out of your way to write something like
let hash = match result {
OK(s) => s,
Err(s) => s,
};
but at that point I feel like the library is forgiven (except for the poor error design).
@my-huge-pony does make a good point elsewhere in the thread: if the potential error was not foreseen and the code was written originally to return `String` instead of `Result<String, Error>` then even Rust would be in a pickle.
it's unbelievable to me that there isn't a larger movement of developers to move away from c towards better & safer languages. look at how much pushback the rust4linux folks get, primarily from c developers who are simply afraid of change. imagine engineers not adopting safer building practices because "they prefer the old way"....it's disgusting honestly.
what will it take to move away from c? older devs retiring. Planck said that "science advances one funeral at a time" and i fear it will be the same for software.
Moving away from C in any POSIX system is in fact stop being POSIX system as POSIX and linux being unix like system is built on standard that is defined as C library specification.
Rewriting it to Rust is fine attempt, just hard to call it is unix/posix anymore. It may not matter but semantically true.
Cars without seatbelts and airbags chopping off driver's heads in relatively slow collisions? Well guess what, that could happen in a modern car as well if the driver sticks their head out the window while going through a very tight tunnel!
It's nice to see how quickly this is being addressed, and likely monitored for any unintended side effects.
Looking at the unpatched Ubuntu things in my life, and I'm pretty close to asking Claude to update things to see how quickly Alpine can get close, assuming responsiveness can stay this fast and quality.
> set PasswordAuthentication no in your sshd configuration and restart sshd.
Funnily enough, they don’t recommend checking pubkey auth is enabled and working. By blindly (or agenticly) following those instructions the risk is getting locked out of some machine.
I agree entirely, but I'm not sure changing `crypt` to return `"*"` instead of `NULL` on failure qualifies. It's much more of a considered decision, which happens to be very wrong.
Except in the case where their images are built with OpenSSH instead of Dropbear, which is the case for some manufacturers. (Turris, was noted as one such vendor)
It didn't give me the impression that I was dealing with a secure piece of software, it seemed like they were dealing with very complex issues while laughably failing at the industry basics that were settled years ago.
Similarly Ubuntu was also the only other distro that fucked this simple permissions config up, and lately with the latency to fix the TCP vuln, it seems to confirm that these are not reliable distros for the coming security era.
both alpine and musl are great in my opinion, and as far as I know theres no empirical evidence that it is more or less secure overall than alternatives. would you like to elaborate your opinion?
I have to admit that I am a bit disappointed in this kind of thinking. There's vulnerabilities everywhere as especially the past few weeks have shown.
One occurrence is not a statistic and should not be used as evidence - it's just an example of a vulnerability that due to various bad circumstances lead to a particularity bad outcome. This logic is in my opinion equally flawed as calling alpine safe now because this particular vulnerability has been fixed.
I'm not a big fan of blaming and I think it's just a result of bad circumstances, but it's interesting to see how some seem to blame musl, but one could argue further that musl documented the behavior of their API which was ignored by openssh and so on
IMO only good thing about alpine is that it's a good environment to build stuff that links musl statically.
Never understood why Alpine is used to actually run anything in a container - if you statically link binary, then you don't need alpine as a base and if you don't what is the point? Don't @ me about image size, even without layer caching we're talking about tiny difference that is often dwarfed by size of your runtime.
Musl itself is a great example of taking "simple and lightweight" so seriously that everyone else gets to deal with the complexity. Slow allocator, underoptimized memcpy/string routines, limited locale support, no NSS, and endless glibc compatibility headaches.
But hey, at least libc is small. %%Proceeds to pull in megabytes of <any other>malloc, and other libraries to replace the slow parts of musl.%%
And now sshhole.com is a particularly spectacular example of what happens when "simple" meets the rest of the software ecosystem.
Static linking makes distribution easier, but it's an absolute security nightmare.
With dynamic linking, you just upgrade the library on your system and you're done. With static linking, every binary that uses the library has to be rebuilt.
Seems slightly confusing that the OpenSSH release notes page says:
> This release contains a critical security workaround that affects Linux systems that use the musl libc and PAM disabled.
But that there's nothing about it on their security page.
Even if it's not technically a bug in their code, the fact that they've made a "critical" release to address it suggests that it should probably be listed on the security page as well. But maybe (hopefully) they just haven't updated the security page yet.
musl libc violates the POSIX documented behavior of `crypt()`¹
> Upon successful completion, crypt() shall return a pointer to the hashed password; the first two bytes of the returned value shall be those of the salt argument. Otherwise, it shall return a null pointer and set errno to indicate the error.
OpenBSD is not affected, so apparently they are doing something right. But also glibc linux is not affected and pam linux is not affected, so it is only musl linux with the remote hole.
You became a victim of a very misleading post title. This isn't an OpenSSH problem at all, it's a musl problem.
I'm pretty sure the misleading wording is deliberate, OpenSSH is an excellent but non-commercial project which a bunch of entities would love to gain control of, in order to bug it with their spyware - thus the constant nagging on the bot-infested web.
Does anyone know why musl does this? Are they the only C library that behaves this way?
For a C library that famously tries to adhere to POSIX as much as possible, this seems like a crazy choice. From POSIX standard:
Upon successful completion, crypt() shall return a pointer to the encoded string. The first two bytes of the returned value shall be those of the salt argument. Otherwise, it shall return a null pointer and set errno to indicate the error.
https://pubs.opengroup.org/onlinepubs/9699919799/functions/c...