Thursday, July 7, 2016

Java cacerts on Intel Edisons and Gateways

The Intel Edison ships with loads of ca certificates in /etc/ssl/certs, and it also ships with Java 8 (OpenJDK), but it does not ship with a pre-configured cacerts file for Java. Ditto for the Wind River Linux installation that is preinstalled on the Dell 3290 gateway device.

So if you try to use Java to access something over HTTPS you'll get a security exception. For example, if you want to run Clojure using the excellent boot build too, the first time you try "$ boot repl" on either WRLinux/Gateway or Yocto/Edison, you’re going to get an exception along the lines of:

Exception in thread "main"
java.lang.RuntimeException: Unexpected error:
the trustAnchors parameter must be non-empty

The problem is that although both systems ship with a large set of ca certificates, as well as preinstalled OpenJDK 1.8, the latter is not configured to use the former out of the box.  Since I virtually never have to deal with this sort of thing it took me a couple of hours to figure out what the problem was and how to fix it.  All you have to do is create a Java “trust store” by running a Java utility calledkeytoolonce for each certificate you want to add to the trust store. (A trust store is analogous to a key store; the latter is where you keep your keys, the former is where you keep public certificates of trust; if you’re a glutton for punishment take a look at Configuring Java CAPS for SSL Support.)
Here’s what you need to know to make sense of that:
  • Java is installed at /usr/lib64/jvm (WRLinux on the Gateway), or /usr/lib/jvm (Yocto on Edison)
  • The standard place for the Java truststore is $JAVA_HOME/jre/lib/security/cacerts. Note that cacerts is a file, not a directory. You create it with
  • keytool
  • keytool is located in $JAVA_HOME/jre/bin
  • The certificates are in /etc/ssl/certs, which is a directory containing soft links to /usr/share/ca-certificates

Now you just need something to save you the drudgery on installing everything in /etc/ssl/certs into the Java trust store, since keytool must be given an individual file rather than a directory of files as input. Fortunately, somebody already wrote that script for us. You can find it at Introduction to OpenJDK in the subsection Install or update the JRE Certificate Authority Certificates (cacerts) file. I just copied that to ~/bin/mkcacerts, chmodded it to make it executable, and then created a one-time helper:


# use this for Yocto/Edison:
# LIB=lib
# use this for WRLinux/Gateway
if [ -f /usr/$LIB/jvm/java-8-openjdk/jre/lib/security/cacerts ]; then
  mv /usr/$LIB/jvm/java-8-openjdk/jre/lib/security/cacerts \
# if you have a ca-certificates.crt file, use this:
# -f "/etc/ssl/certs/ca-certificates.crt"
# otherwise use
# -d "/etc/ssl/certs/"
./mkcacerts                 \
        -d "/etc/ssl/certs/"           \
        -k "/usr/$LIB/jvm/java-8-openjdk/bin/keytool"      \
        -s "/usr/bin/openssl"          \
        -o "/usr/$LIB/jvm/java-8-openjdk/jre/lib/security/cacerts"

There is one tricky bit: notice the "-d" parameter in the script above. Use that if /etc/ssl/certs contains certificates but no "ca-certificates.crt" - this was the case in earlier versions of the Edison. But if you find /etc/ssl/certs/ca-certificates.crt, then replace that line as noted in the comment.

Now cd to ~/bin and run $ ./  You'll see a bunch of alarming-looking output as it churns through the certificates, adding them to the Java truststore (cacerts file). It takes a while, but once it's done you should be good to go with Java and HTTPS.

Tuesday, July 5, 2016

Why I Still Hate Hate Hate Build Systems

You might think that by 2016 software and systems developers would have figured out build systems. But no.  Build systems, along with package management and configuration, remain the bane of the developer's, not to mention the user's, existence.

I've worked with many such systems, from the venerable make/autotools system, to Cmake, to Scons, to Ant, Maven, Gradle (Boo!) and who knows what else. I hate them all.  They are all misbegotten monstrosities that only their mothers could love.

Case in point: I want to use the tinyb Bluetooth LE library on the Intel Edison and on a Dell 3290 gateway device.  You'd think that would be easy enough - download, configure, make, install, use. The project uses CMake, which ought to be even better than Make.

But like all build systems, Cmake works just great - until it doesn't.  In this case:

  • It started out by offering an inscrutable error message:  "A required package was not found". I'm not making this up, that is the message.  Which package?  Why so coy, CMake?  Cantcha just say???
  • So you look in the error log, and you see that a check for Pthreads failed. But you know your system has pthreads, so you flail around for a while, then notice it failed because the test command specified not "-lpthread", but "-lpthreads".  Whaa? A bug in CMake? An hour and a half later you finally realize that was a spurious report - the error message was not caused by Pthreads testing but by the check that occurred just before the error message. You proceed to feel both very stupid, because you should have thought of that immediately, and very, very pissed that the morons in charge of Cmake could have saved you the trouble by simply saying what the problem is.  I dunno, maybe something like "A required package was not found: gio-unix-2.0>=2.40"?  Izzat so hard?
  • And the way you actually figure out that gio-unix-2.0 was the problem is by creating a test project and painstakingly going through the CMakeLists.txt file line by line, trial-and-error. You begin to fear you might have an aneurism.  You are glad the Cmake devs are not within throttling distance, lest you squeeze their throats very hard.
  • But, but, bu bu - my system does so have gio-unix-2.0!  It's right there! (Of course, you waste a good 20-30 minutes googling around trying to figure out what exactly gio-unix is).
  • More trial and error.  Maybe its the version number.  You try a bunch, nothing makes a difference, the test always fails.
  • Finally, you inspect the pkg-config file for gio-unix-2.0. Everything looks hunky dory. Except for that @[email protected] thingie there, it looks a little fishy, but you are not a pkg-config expert, you assume that it must be there for a reason.  But then you look at the related gio-2.0 file, and there in the version field is an actual number!  2.48.1, to be exact!
  • So finally it dawns on you: the person whose neck you need to squeeze, hard, mercilessly, is the person who install the broken pkg-config file for gio-unix-2.0, the one that didn't get the version number so it fails all checks.
  • But the CMake devs do not get away so easily!  It's still a flaw in their product - when they test for a library with a version >=2.40, and the version they get is obviously broken - not a version number at all - they should throw a "broken config file" exception, not a "package not found" error.  It's just false to say the package was not found! And it would also be false to say that the package found had too low a version number! It did not fail the version test, it was incapable of even taking the version test, and that is the exception.

The real problem with build systems is that they do not treat the build engineering problem as a software development problem. To manage builds you need a real programming language, and you need to write your build scripts with all the care you would use in writing critical C code.  There is actually one project out there that takes this approach, the only one I know of: Whereas previous build systems are of the "build a better mousetrap" variety, so they're all pretty much the same thing, boot represents a fairly radical re-conceptualization of what building software is all about.


Both the Intel Edison and the Dell 3290 (Intel IoT Gateway) with Wind River Linux, being embedded systems, run a Yocto-based OS.  The package manager for Yocto is opkg, not apt nor rpm nor any of the other PMs one typically encounters in full-featured Linux environments.  Unfortunately, good information on how to correctly use opkg is not easy to find.  Here's what worked for me:

Here "Edison" means an Intel Edison module updated to the latest image as of June 2016, and "Gateway" means a Dell/Wyse 3290 running Wind River Linux version 6.  Note that the Intel IoT Gateway platform also supports Ubuntu and Windows OSs.

opkg homepage:


Config file:

  • Edison:
  • Gateway:  /etc/opkg/opgk.conf
Here's what I used on the Gateway:

# opkg.conf

dest root /
dest ram /tmp
lists_dir ext /var/opkg-lists
option overlay_root /overlay
src/gz x86_base
src/gz x86_packages