>>kkl+(OP)
I heard that Qubes is riddled with undocumented scripts that have no automated tests for them. Does anybody know how true is that?

>>d33+93
Qubes is open source[1] (including the Windows capabilities[2]) so your question should be fairly easy to prove/disprove yourself.

I haven't delved into the source of any Qubes scripts or unit tests, but if one were to act on the rumours you heard perhaps they'd start with the Qubes Builder Scripts referenced in the article. If the Qubes team is able to implement a true append-only logging for their development+build process then perhaps any concerns around undocumented, unknown or untested scripts could be somewhat assuaged.

[1] https://github.com/QubesOS/

[2] https://www.qubes-os.org/news/2016/01/27/windows-tools-open-...

>>kkl+(OP)
Rutkowska is surely doing some of the most interesting research in this field.

>>kkl+(OP)
Their build process sounds similar to Bitcoin Core's release process.

Bitcoin downloads dependencies, checks them against their preconfigured hashes and then builds the different versions (Windows, Linux, Mac) in different VMs. The build is deterministic and thus produce the exact same files for everybody. Everyone signs the files they produced and uploads the signature. If all the signatures are valid for the same file you can be reasonably sure that the build process wasn't tampered with.

Getting deterministic builds even for a project like Bitcoin Core with few dependencies was hard. On the scale of Qubes this would be a monumental task. But maybe Debian's initative for reproducible builds makes this easier in the future.

>>d33+93
Just in case, I'm reposting my message from https://news.ycombinator.com/item?id=11652940

On the workstation part, it recommends QubesOS. Am I the only one who is skeptical about it?

From what I saw superficially reading their source code, there are some frightening stuff going on:

* tons of C code with nearly zero unit tests, same with the python code

* lots glue in form of bash or python scripts

* some not so beautiful stuff like:

- https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (kill -9 on a daemon...)

- https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (a daemon is a little bit more than an exe launched with '&'

- https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (changing a config file in an init script, humm, weird...)

- https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... (starting a service inside the init of another service...)

- https://github.com/QubesOS/qubes-core-agent-linux/blob/maste... ("logging" with stderr redirection in a file)

And it's just the init scripts... I'm too lazy to take a look further inside the C or python stuff. IMHO, as a proof of concept, it's interesting, as a finished, reliable and secure OS, it's frightening...

>>kakwa_+in
Ah, yes, it's your comment that I was referring to.

>>wongar+L8
To what degree are the Bitcoin build VMs reproducible vs. giant binary blobs?

>>d33+93
I think this is an important point based on the following taken from the article:

our script for the verify-sources target in the Xen component contained a silly bug that resulted in the script not exiting upon failure

>>gaius+Z7
Rutkowska is among the crème de la crème of computer security researchers. That she's dedicating her time towards a project like Qubes is sorely under-appreciated by the public today, but is certainly something that inspires me to believe that, given enough time, defenders can win.

>>CiPHPe+0y
I'll give her credit for putting the time in more than most. Even I'm guilty on that one as my brain-damaged ass could be cloning UNIX apps in Rust or SPARK or something with MAC or Capsicum policies. My research is important but most of us doing such things don't do enough OSS coding.

>>gaius+Z7
Barely. Most of her work is behind what was done in the 80's and 90's for security kernels then 2000's for separation kernels. I criticized her for not building on proven foundations and methods. She censors my stuff but did eventually say and do some of same things. My Xen gripes, GUI trusted path... these come to mind.

For an example, here's an Orange Book A1-class VMM by legend Paul Karger. He's one of inventors of INFOSEC, genius designer/coder, and high-assurance veteran. Look at the design and assurance sections (p9 onward) of it to see what... in 90's... was necessary to secure a VMM via minimal privilege (POLA), correctness arguments, backdoor prevention, and covert channel suppression. Nothing today in OSS even has this baseline despite us discovering more problems and solutions. Re-reading it now, I noticed they were even doing continuous integration on it well before that became a fad.

http://lukemuehlhauser.com/wp-content/uploads/Karger-et-al-A...

A modern example, one I cited on their mailing list, is INTEGRITY-178B. The features plus assurance activities are a nice illustration of high-assurance approach to microkernels for security or virtualization vs things like Xen. Quite a few things worth copying for security- or reliability-focused OSS projects. Approaches that got open-sourced from CompSci are in links below it.

http://www.ghs.com/products/safety_critical/integrity-do-178...

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=9EE...

http://genode-labs.com/publications/nfeske-genode-fosdem-201...

Note: GenodeOS is a competitor that uses components like above with architecture designed to lower risk in TCB like in high-assurance. It's nice work. Fundamental architecture needs peer review, though, to ensure it has claimed properties.

>>kkl+(OP)
Anyone interesting in securing repo's or build systems should start with Wheeler's landmark collection on the topic:

http://www.dwheeler.com/essays/scm-security.html

Has basics in English, CompSci work, high-assurance considerations, and some example projects. A bright, security researcher that's very familiar with DVCS's should redo this in light of them with similar recommendations. More like a team of bright researchers but it needs to be done. I'm interested in any papers people already have on this that have similarly-thorough treatment of threat model and proposed mitigations.

Once you know builds, you might want to address subversion, design, implementations, covert channels, and other things if you're trying to stop Five Eyes, Russia, or China. That requires "high-assurance" security methods... when it's even possible... Got a small list here to get people started on how deep the issue goes just at high-level and subversion aspects:

https://news.ycombinator.com/item?id=10478742

>>kakwa_+in
It's hall-marks of a low-assurance, software project hacked together to get it working with OSS components. There could be strong review in scripts or C code as the team are talented coders and breakers from what I've read. The minimum in high-assurance security... even medium-assurance... was that every feature was tested for proper behavior during successful execution and failures. Plus, a description of why it's there that traces to a functional or security requirements. That's so evaluators can spot unnecessary code that might be a backdoor or just dead code.

Python for those scripts is questionable, too, as it's too complex to analyze with known principles of security engineering. It should be a 3GL that maps well to hardware with easy way to spot code-level flaws or automated them out (esp static analysis). You can abstract it, even macro it, so long as final code can be subjected to whatever eyeballs or tools are out there with ease. Non-security-critical stuff can be written in something like Python, though.

>>j_s+Nv
The build VMs are predefined Ubuntu LTS releases. The build system that sets everything up is a bunch of barely documented scripts.

>>nickps+uG
I think the reason Qubes is interesting is not because it's at the forefront of theoretical security, but that it's actually useable today as a desktop. None of the systems you mentioned meet that criteria.

Using a computer today requires interoperating with such a bewildering array of other systems. Just writing a web browser is a huge undertaking.

It's wrong to compare Qubes to academic microkernels that require applications written in a formal language. It should be compared to a general Linux/BSD distro or to Windows, because those are the systems it's competing for users with. In comparison to those, it's a much more solid platform for security.

>>throwa+Yq1
You could've said same thing about Qubes in early state that you said about "academic" systems. Both just required work to get in usable shape. Far as comparisons, I do two types: compare it to mainstream OS's as you said; compare it to other systems in its category.

Now, unlike your claim, others were in production under label MILS systems far back as 2005. They used separation kernels to host VM's for Linux and Windows with networking, filesystem, GUI, etc in separate partitions plus color labels on screens. Sound familiar? Additionally, the Turaya work in Europe got turned into commercial products from Sirrix. OKL4's was deployed in a billion phones. Genode's tiny team has made theirs quite usable in short time despite all the custom work done.

So, Qubes wasn't the first, most polished, most secure, least academic, or anything. It's a latecomer using inherently bad components but with high usability and tolerance to regular malware. There's an upper limit to how much security if can provide as malware sophistication and threat model increases. So, I encourage its use only for lower, threat profiles like average user browsing the web with investments into stronger architectures for higher, risk use.

>>throwa+Yq1
Genode is useable as a desktop. Use the Nova kernel.

No binaries are distributed for obvious reasons. You can setup a build environment very quickly. It's simple stuff. See the Genode book: http://genode.org/documentation/genode-foundations-16-05.pdf

Oh, check out the rump kernel integration afterwards.

Fyi, package manager integration via Nix is in the works.