CAs now get to decide who’s on the Internet

It started with a legit concern. The majority of websites were served using HTTP. HTTP is insecure. So what you might be thinking? HTTPS is used on my bank and amazon and anywhere I might spend money so it seems not a problem. Except … HTTP allows injections. Ever use some bad hotel or bad airport WiFi and get a banner injected at the top of the screen? That’s HTTP vs HTTPS. Are you sure those articles you’re reading are the originals? Maybe someone is changing words, pictures, or ads. HTTPS solves these issues.

So, the browser vendors and other standards bodies got together and made a big push for HTTPS only. Sounds great right!?

Well instead of just pushing metaphorically by putting out the word, “Stop using HTTP! Start using HTTPS” the browser vendors got together and decided to try to kill off HTTP completely. Their first order of business was to start requiring HTTPS to use certain features in the browser. Want to go fullscreen? Your site must be served from HTTPS. Want to read the orientation and motion data of the phone from the browser? Your website must use HTTPS. Want to be able to ask the user for permission to access to mic or the camera? Your website must use HTTPS.

Okay well that certainly can be motivating to switch to HTTPS as soon as possible.

Except … HTTPS requires certificates. Those certificates can only be acquired from Certificate Authorities. CAs for short. CAs charge money for these certificates. $50 per certificate or more. Often the certificates only last for a limited time so you’ve got to pay every year or 2.

Suddenly every website just got more expensive.

Okay you say but that’s still not a ton of money.

Yes but, maybe you’ve got an innovative project. One that lets any user access their media from their browser(example). You’d like to let them go fullscreen but you can’t unless it serves the media pages as HTTPS. The rules of HTTPS say your not allowed to share certs ever. If you get caught sharing your cert will be invalidated. So, you can’t give each of these people running your innovated software a copy of your cert. Instead every user needs their own cert. Suddenly your software just got a lot more expensive! What if your software was free and open source? In 2015 people were able to run it for free. In 2016 the are now required to get a cert for $50

So what do you do? Well you hear about self-signed certs. So you check those out. Turns out they require complex installation into your OS. Your family and aunts and uncles and cousins and nephews and nieces aren’t going to find that really manageable. And besides there’s the feature where anyone can come to a party at your places and queue some music videos using their phone’s browser but that’s never going to fly if they have to first install this self-signed cert. Official certs from CAs don’t have this issue. They just work.

Okay well you shop around for CA’s. Dear CA#1 will you give my users free certs? No! Dear CA#2 will you give my users free certs? No!

Oh I hear you say, there’s a new kid on the block, letsencrypt, they offer free certs.

They do offer free certs BUT, certs are tied to domain names. To get a cert from letsencrpyt you have to have a domain. Example “mymediastreamer.org”. So even if you can get the cert for free your users now need to buy a domain name. That can be relatively cheap at $10-$20 a year but it’s a big technical hurdle. Your non-tech family members are not really going to be able to go through all the process of getting a domain name just to use your media server.

Oh I hear you say, what if my software ran a public DNS server. I could issue users subdomains like “.mymediastreamer.org”. Then I can give out DNS names to the users and they can get certs. That might work … except, DNS points to specific IP addresses. User’s IP address changes. You can re-point DNS to the new address but it takes time to propagate. That means when their IP address changes it might be a few hours until they can access their media again. Not going to work.

Ok then here’s a solution. We’ll make up domains like this “..mymediastreamer.org”. That will make the DNS server even easier. We don’t even need a database. We just look at the “” part of the DNS name and return that IP address. Now when the user’s IP address changes there will be zero delay because they can immediately use a DNS name that matches. We’ll setup some rendezvous server for them so they don’t need to lookup the correct domain. It will all just work.

Great! We have domains. We can get free certs from letsencrypt.

Except….letsencrypt limits the number of certs to 240 per root domain. So once you have 240 you can’t get more certs. That means we can only support 240 users at best. But then there’s another problem. Letsencrpt doesn’t support wildcard certs. Because we added the part above we need a wildcard cert *for each user* matching “*..mymediastreamer.org”.

Effectively we are S.O.L. For our purposes letsencrypt is just another CA. “CA#3 can we please have free certs for our users?” No!

As of 2015 we could do anything we wanted on the internet. Now in 2016 we need permission from a CA. If the CA doesn’t give permission we don’t get on the internet.

To put it another way because of the chain of validation in HTTPS each CA is effectively a little king/bureaucrat who gets to decide who gets on the internet and who doesn’t. If one king doesn’t let you your only option is to go ask another king. Letsencyrpt is the most generous king as they don’t ask for tribute but that doesn’t change the fact you still need permission from one of these kings.

You might be thinking, “so what? who cares about a media streamer?”. But it’s not just streamers. It’s “ANY DEVICE OR SOFTWARE THAT SERVES A WEBPAGE”. Got an IP camera that serves a webpage? That camera wants to give you a nice interface that goes fullscreen? It can’t without certs and it can’t get certs without permission from a CA. Got some Raspberry PI project that wants to serve a webpage and needs any of the banned features? Again, it can’t do it without a cert and it can’t get a cert without permission from a CA. Maybe you have a NAS device and it would like provide web page access? It can’t do it without a cert and it can’t get certs without permission from a CA.

That wasn’t the case just 6 months ago because HTTPS wasn’t required. Now that it is these kings all just got a bunch more power and innovative products like the media streamer described above and projects like this are effectively discouraged unless you can beg or bribe a king to ordain them. 🙁

More NPM BS

Sorry to rant on this more but sheesh! How much does it take?

So I want to watch the filesystem for changes. Unfortunately node’s fs.watch isn’t OS independant. Some OSes return individual files. Other OSes just return the parent folder changed. I’m sure there’s some legit reason like perf not to fix this for all devs but whatever. I assume there must be a library that fixes it.

So I search NPM. Maybe I should be using Google to search NPM because NPMs results are usually pretty poor but after searching a while I find this watch module. It’s got nearly 700k downloads in the last month and tons of dependents so it must be good.

Given my prior experience the first thing I check is how many dependencies does it list?

Okay only 2 but they are both related to command line stuff. WTF does a library have command line dependencies? Oh, I see this library is conflated with a utility that watches for changes and launches a specified program anytime a file changes. Sigh… 🙄

Okay well let’s look through the dependencies. It’s not too deep. Fine, maybe I’ll live with it if it works.

Ok I file an issue “Separate library from command line?” and move on.

A quick glance at the code and it doesn’t appear to abstract the issues I need fixed but it’s easy to test. So I write some test code

var watch = require('watch');

var dir = process.argv[2];
console.log("watching: ", dir);

var options = {
  ignoreDotFiles: true, // - When true this option means that when the file tree is walked it will ignore files that being with "."
  filter: function() { return true; }, // - You can use this option to provide a function that returns true or false for each file and directory to decide whether or not that file/directory is included in the watcher.
  ignoreUnreadableDir: true, // - When true, this options means that when a file can't be read, this file is silently skipped.
  ignoreNotPermitted: true, // - When true, this options means that when a file can't be read due to permission issues, this file is silently skipped.
  ignoreDirectoryPattern: /node_modules/, // - When a regex pattern is set, e.g. /node_modules/, these directories are silently skipped.
};

watch.createMonitor(dir, options, function(monitor) {
  monitor.on('created', function(f, s)     { show("created", f, s    ); });
  monitor.on('removed', function(f, s, s2) { show("removed", f, s, s2); });
  monitor.on('changed', function(f, s)     { show("changed", f, s    ); });
});

function show(event, f, s, n) {
  console.log(event, f);
}

I run it and pass it my temp folder which has a ton of crap in it including lots of sub folders.

I edit temp/foo.txt and see it print changed: temp/foo.txt. I delete temp/foo.txt and see it print removed: temp/foo.txt. I create a temp/foo.txt and see it print created: temp/foo.txt.

So far so good. Let’s test one more thing.

I look for a subfolder since I’ve only been changing stuff in the root of the folder I’m watching. I just happen to pick temp/delme-eslint/node_modules/foo. BOOM! It prints

created temp/delme-eslint/node_modules/abab
created temp/delme-eslint/node_modules/abbrev
created temp/delme-eslint/node_modules/acorn
created temp/delme-eslint/node_modules/acorn-globals
created temp/delme-eslint/node_modules/acorn-jsx
created temp/delme-eslint/node_modules/amdefine
created temp/delme-eslint/node_modules/ansi-escapes
created temp/delme-eslint/node_modules/ansi-regex
created temp/delme-eslint/node_modules/ansi-styles
created temp/delme-eslint/node_modules/argparse
created temp/delme-eslint/node_modules/array-differ
created temp/delme-eslint/node_modules/array-union
created temp/delme-eslint/node_modules/array-uniq
created temp/delme-eslint/node_modules/arrify
created temp/delme-eslint/node_modules/asn1
created temp/delme-eslint/node_modules/assert-plus
created temp/delme-eslint/node_modules/async
created temp/delme-eslint/node_modules/aws-sign2
created temp/delme-eslint/node_modules/aws4
created temp/delme-eslint/node_modules/balanced-match
created temp/delme-eslint/node_modules/bl
created temp/delme-eslint/node_modules/bluebird
created temp/delme-eslint/node_modules/boolbase
created temp/delme-eslint/node_modules/boom
created temp/delme-eslint/node_modules/brace-expansion
created temp/delme-eslint/node_modules/caller-path
created temp/delme-eslint/node_modules/callsites
created temp/delme-eslint/node_modules/caseless
created temp/delme-eslint/node_modules/chalk
created temp/delme-eslint/node_modules/cheerio
created temp/delme-eslint/node_modules/cli-cursor
created temp/delme-eslint/node_modules/cli-width
created temp/delme-eslint/node_modules/code-point-at
created temp/delme-eslint/node_modules/coffee-script
created temp/delme-eslint/node_modules/color-convert
created temp/delme-eslint/node_modules/colors
created temp/delme-eslint/node_modules/combined-stream
created temp/delme-eslint/node_modules/commander
created temp/delme-eslint/node_modules/concat-map
created temp/delme-eslint/node_modules/concat-stream
created temp/delme-eslint/node_modules/core-util-is
created temp/delme-eslint/node_modules/cryptiles
created temp/delme-eslint/node_modules/css-select
created temp/delme-eslint/node_modules/css-what
created temp/delme-eslint/node_modules/cssom
created temp/delme-eslint/node_modules/cssstyle
created temp/delme-eslint/node_modules/d
created temp/delme-eslint/node_modules/dashdash
created temp/delme-eslint/node_modules/dateformat
created temp/delme-eslint/node_modules/debug
created temp/delme-eslint/node_modules/deep-is
created temp/delme-eslint/node_modules/del
created temp/delme-eslint/node_modules/delayed-stream
created temp/delme-eslint/node_modules/doctrine
created temp/delme-eslint/node_modules/dom-serializer
created temp/delme-eslint/node_modules/domelementtype
created temp/delme-eslint/node_modules/domhandler
created temp/delme-eslint/node_modules/domutils
created temp/delme-eslint/node_modules/ecc-jsbn
created temp/delme-eslint/node_modules/entities
created temp/delme-eslint/node_modules/es5-ext
created temp/delme-eslint/node_modules/es6-iterator
created temp/delme-eslint/node_modules/es6-map
created temp/delme-eslint/node_modules/es6-set
created temp/delme-eslint/node_modules/es6-symbol
created temp/delme-eslint/node_modules/es6-weak-map
created temp/delme-eslint/node_modules/escape-string-regexp
created temp/delme-eslint/node_modules/escodegen
created temp/delme-eslint/node_modules/escope
created temp/delme-eslint/node_modules/eslint
created temp/delme-eslint/node_modules/espree
created temp/delme-eslint/node_modules/esprima
created temp/delme-eslint/node_modules/esrecurse
created temp/delme-eslint/node_modules/estraverse
created temp/delme-eslint/node_modules/esutils
created temp/delme-eslint/node_modules/event-emitter
created temp/delme-eslint/node_modules/eventemitter2
created temp/delme-eslint/node_modules/exit
created temp/delme-eslint/node_modules/exit-hook
created temp/delme-eslint/node_modules/extend
created temp/delme-eslint/node_modules/extsprintf
created temp/delme-eslint/node_modules/fast-levenshtein
created temp/delme-eslint/node_modules/figures
created temp/delme-eslint/node_modules/file-entry-cache
created temp/delme-eslint/node_modules/find-up
created temp/delme-eslint/node_modules/findup-sync
created temp/delme-eslint/node_modules/flat-cache
created temp/delme-eslint/node_modules/foo
created temp/delme-eslint/node_modules/forever-agent
created temp/delme-eslint/node_modules/form-data
created temp/delme-eslint/node_modules/generate-function
created temp/delme-eslint/node_modules/generate-object-property
created temp/delme-eslint/node_modules/getobject
created temp/delme-eslint/node_modules/glob
created temp/delme-eslint/node_modules/globals
created temp/delme-eslint/node_modules/globby
created temp/delme-eslint/node_modules/graceful-fs
created temp/delme-eslint/node_modules/graceful-readlink
created temp/delme-eslint/node_modules/grunt
created temp/delme-eslint/node_modules/grunt-eslint
created temp/delme-eslint/node_modules/grunt-legacy-log
created temp/delme-eslint/node_modules/grunt-legacy-log-utils
created temp/delme-eslint/node_modules/grunt-legacy-util
created temp/delme-eslint/node_modules/har-validator
created temp/delme-eslint/node_modules/has-ansi
created temp/delme-eslint/node_modules/hoek
created temp/delme-eslint/node_modules/hawk
created temp/delme-eslint/node_modules/hooker
created temp/delme-eslint/node_modules/htmlparser2
created temp/delme-eslint/node_modules/http-signature
created temp/delme-eslint/node_modules/iconv-lite
created temp/delme-eslint/node_modules/ignore
created temp/delme-eslint/node_modules/inflight
created temp/delme-eslint/node_modules/inherits
created temp/delme-eslint/node_modules/inquirer
created temp/delme-eslint/node_modules/is-fullwidth-code-point
created temp/delme-eslint/node_modules/is-my-json-valid
created temp/delme-eslint/node_modules/is-path-cwd
created temp/delme-eslint/node_modules/is-path-in-cwd
created temp/delme-eslint/node_modules/is-path-inside
created temp/delme-eslint/node_modules/is-property
created temp/delme-eslint/node_modules/is-resolvable
created temp/delme-eslint/node_modules/is-typedarray
created temp/delme-eslint/node_modules/isarray
created temp/delme-eslint/node_modules/isstream
created temp/delme-eslint/node_modules/jodid25519
created temp/delme-eslint/node_modules/js-yaml
created temp/delme-eslint/node_modules/jsbn
created temp/delme-eslint/node_modules/jsdom
created temp/delme-eslint/node_modules/json-schema
created temp/delme-eslint/node_modules/json-stable-stringify
created temp/delme-eslint/node_modules/json-stringify-safe
created temp/delme-eslint/node_modules/jsonify
created temp/delme-eslint/node_modules/jsonpointer
created temp/delme-eslint/node_modules/jsprim
created temp/delme-eslint/node_modules/levn
created temp/delme-eslint/node_modules/load-grunt-tasks
created temp/delme-eslint/node_modules/lodash
created temp/delme-eslint/node_modules/lru-cache
created temp/delme-eslint/node_modules/mime-db
created temp/delme-eslint/node_modules/mime-types
created temp/delme-eslint/node_modules/minimatch
created temp/delme-eslint/node_modules/minimist
created temp/delme-eslint/node_modules/mkdirp
created temp/delme-eslint/node_modules/ms
created temp/delme-eslint/node_modules/multimatch
created temp/delme-eslint/node_modules/mute-stream
created temp/delme-eslint/node_modules/node-uuid
created temp/delme-eslint/node_modules/nopt
created temp/delme-eslint/node_modules/nth-check
created temp/delme-eslint/node_modules/number-is-nan
created temp/delme-eslint/node_modules/nwmatcher
created temp/delme-eslint/node_modules/oauth-sign
created temp/delme-eslint/node_modules/object-assign
created temp/delme-eslint/node_modules/once
created temp/delme-eslint/node_modules/onetime
created temp/delme-eslint/node_modules/optionator
created temp/delme-eslint/node_modules/os-homedir
created temp/delme-eslint/node_modules/parse5
created temp/delme-eslint/node_modules/path-exists
created temp/delme-eslint/node_modules/path-is-absolute
created temp/delme-eslint/node_modules/path-is-inside
created temp/delme-eslint/node_modules/pify
created temp/delme-eslint/node_modules/pinkie
created temp/delme-eslint/node_modules/pinkie-promise
created temp/delme-eslint/node_modules/pkg-up
created temp/delme-eslint/node_modules/pluralize
created temp/delme-eslint/node_modules/prelude-ls
created temp/delme-eslint/node_modules/process-nextick-args
created temp/delme-eslint/node_modules/progress
created temp/delme-eslint/node_modules/pseudomap
created temp/delme-eslint/node_modules/qs
created temp/delme-eslint/node_modules/read-json-sync
created temp/delme-eslint/node_modules/readable-stream
created temp/delme-eslint/node_modules/readline2
created temp/delme-eslint/node_modules/request
created temp/delme-eslint/node_modules/require-uncached
created temp/delme-eslint/node_modules/resolve
created temp/delme-eslint/node_modules/resolve-from
created temp/delme-eslint/node_modules/resolve-pkg
created temp/delme-eslint/node_modules/restore-cursor
created temp/delme-eslint/node_modules/rimraf
created temp/delme-eslint/node_modules/run-async
created temp/delme-eslint/node_modules/rx-lite
created temp/delme-eslint/node_modules/sax
created temp/delme-eslint/node_modules/shelljs
created temp/delme-eslint/node_modules/sigmund
created temp/delme-eslint/node_modules/slice-ansi
created temp/delme-eslint/node_modules/sntp
created temp/delme-eslint/node_modules/source-map
created temp/delme-eslint/node_modules/sprintf-js
created temp/delme-eslint/node_modules/sshpk
created temp/delme-eslint/node_modules/string-width
created temp/delme-eslint/node_modules/string_decoder
created temp/delme-eslint/node_modules/stringstream
created temp/delme-eslint/node_modules/strip-ansi
created temp/delme-eslint/node_modules/strip-json-comments
created temp/delme-eslint/node_modules/supports-color
created temp/delme-eslint/node_modules/symbol-tree
created temp/delme-eslint/node_modules/table
created temp/delme-eslint/node_modules/text-table
created temp/delme-eslint/node_modules/through
created temp/delme-eslint/node_modules/tough-cookie
created temp/delme-eslint/node_modules/tr46
created temp/delme-eslint/node_modules/tryit
created temp/delme-eslint/node_modules/tunnel-agent
created temp/delme-eslint/node_modules/tv4
created temp/delme-eslint/node_modules/tweetnacl
created temp/delme-eslint/node_modules/type-check
created temp/delme-eslint/node_modules/typedarray
created temp/delme-eslint/node_modules/underscore
created temp/delme-eslint/node_modules/underscore.string
created temp/delme-eslint/node_modules/user-home
created temp/delme-eslint/node_modules/util-deprecate
created temp/delme-eslint/node_modules/verror
created temp/delme-eslint/node_modules/webidl-conversions
created temp/delme-eslint/node_modules/whatwg-url-compat
created temp/delme-eslint/node_modules/which
created temp/delme-eslint/node_modules/wordwrap
created temp/delme-eslint/node_modules/wrappy
created temp/delme-eslint/node_modules/write
created temp/delme-eslint/node_modules/xml-name-validator
created temp/delme-eslint/node_modules/xregexp
created temp/delme-eslint/node_modules/xtend
created temp/delme-eslint/node_modules/yallist

BUG!!! Really, I used it for 5 minutes ran into bug and yet 700k downloads and a ton of dependents are using this buggy library. Sigh….🙄

Now the question is (a) fix it or (b) write my own.

Yes I know there are good arguments for fixing it. Just last month 700k downloads = 700k fixed downloads next month probably. But of course it possible means arguing with the author, jumping through their hoops. And, I still don’t know if it solves the problem I want solved.

On top of that I there’s at least one thing I wanted to re-design I think. Not only do I want the tree to be watched, I want to know what’s in the tree. It kind of seems like that should be one operation. In other words I don’t want to call some other function to scan the tree to get the list of all things in it and then separately call another function to watch the tree. If I do that then a file that’s created between the first scan and the watch will never show up. It will also be twice as slow AFAICT as the the watch has to scan as well. So, now yet another discussion to be haggled over if I decide to fix vs do it myself.

The worst part is the thing I wanted to do I thought would take all of 20 minutes if that functionality I was looking for existed. Now will also probably take 8x as long to fix vs do it myself on top of writing the feature I was originally setting out to d. Probably not fair but (a) no negociation and (b) I feel more obligated to right tests for fixing but not for my own code 😂

Update:

I filed a bug on that issue but I didn’t purse fixing it. Instead I wrote my own. It took about 20hrs of solid work. Probably only a 2 hours to write but 18 hours to test across 3 platforms and learn new issues. I found out ubuntu, osx, windows, and travis-ci’s containers all behave differently in this area. I’m sure I didn’t catch all the stuff and I still would like to write another 4-8 hours of tests but I need to move on and it’s working so far.

Some things that were interesting is where to make the divisions, what features to put in. For example the most used library I linked to above, the buggy one, has options for ignoring dot files (files that start with .) as well as an option to give it a regular expression to filter (the one that didn’t actually work). After writing mine I found another library that takes complex globs and has debouncing features.

I feel like those features should be layers. It’s probably as little as one line of code to filter. Example:

  var watcher = new SimpleTreeWatcher(pathToWatch, {
    filter: (filepath) => { return path.basename(filepath)[0] !== '.'); },
  });

So why clutter the library with options when doing it this way keeps the library simple and let’s people be as complex or simple as they want. It seems better to just provide a few examples of how to filter than have a bunch of options.

Similarly the debouncing can relatively easily be layered on top. I should add examples to the docs but at least I have something that’s working for me so far.

Dynamic Typing > Static Typing?

I stumbled upon this amazing video called “The Unreasonable Effectiveness of Dynamic Typing for Practical Programs”

It’s about 50mins long and unfortunately vimeo has no fast playback options like youtube so I’d suggest you download it and view it in VLC then click Ctrl+ or Cmd+ to speed up

In any case he makes a very compelling argument that static typing isn’t really all that. His argument is LOOK AT THE DATA!!! Everyone has opinions. People believe static typing catches bugs. People believe static typing helps document code. People believe static typing makes IDEs work better and therefore save time, etc. BUT … those are all just beliefs not backed up by any data. He points out that so far all the studies suggest that’s not true. He claims 5 years ago he was a static type believer but after looking up the research and doing some of his own he’s convinced static typing is a net negative.

You should really watch the talk but I’ll try to sum it up here (more…)

Stack Overflow Attribution Required

So Stack Overflow apparently requires attribution?!??

This has apparently always been the case but it came up recently because they were planning to change the license.

This made me wonder, why? Why should I attribute SO? I certainly don’t want attribution for all my time spend on Stack Overflow. Most of the stuff I look up on Stack Overflow is just the same stuff I could find on MDN or MSDN or some reference manual for some library somewhere but it’s more work to read the entire manual front to back to grok it than it is to just google it and end up getting pointed to the answer on SO. (more…)

More JavaScript vs C#/C++

I ranted before on JavaScript vs C#/C++ and for some reason I was in the mood to revisit that.

Basically the last 2 years I’ve been mostly in JavaScript land. Like most C++ programmers I hated it at first. But now I find it hard to have any desire to switch back. Of course I’ll program in whatever language I need to to do the job. C++ if I’m Unreal. C# if I’m in Unity.

But, kind of like J-Blow mentioned in his first few videos it’s frustrating every time there is busy work to do in C++ or C# that there is not when in JavaScript. To be clear he only mentioned his frustration with C++. AFAICT he has no serious experience with JavaScript. I expect he’s at stage #1 or stage #2 at best (more…)

More Vertexshaderart.com

As fun as it is to make vertexshaderart I’m also tweaking the site

So for example, while this is probably a bad idea as it might end up costing me more money I cleaned things up a little so you embedding is nicer

This one uses the mouse. It’s based on art by paweł which is linked to below.

Figuring out how to get a good response in the visuals so far as been just a guessing game. This one mostly seemed to work once the song really gets started.

This one too

And just trying new stuff

You can see a particular person’s creations for example P_Malin has made some pretty great pieces.

For a given piece you can also see the revisions

And you can mark pieces as private if you want to work on them before letting others see.

I’m trying to figure out how to get more variety on the main page. Originally I had “popular” which was just by likes but the same few things were always at the top. I might do something like aging things so the older they are the more likes or views they need to stay at the top but at the same time I don’t want to see the main page covered with uninteresting stuff like some of mine so I’m not sure what to do there. If you have ideas comment below.