CSS Grid – Fail?

I’m probably just missing the solution but I’m getting the impression CSS Grid really isn’t the right solution and we’ll be asking for another soon.

If you haven’t used CSS Grid it’s way of using CSS to declare that one element is a grid of NxM cells. Almost like a table except CSS Grid happens in CSS. The cool parts are that you can name groups of cells and then you can tell direct children of the grid what group of cells they cover just by name. Because it’s CSS it’s also easy to use media queries to change the layout based on the size of the user’s browser.

That all sounds great, except AFAICT a grid doesn’t actually solve this issue. Here’s an example. Here’s a video about CSS Grid

Note the screenshot for the video itself. In case it gets changed at the time I wrote this post it looked like this

Seeing that thumbnail you’d expect that layout is what you’d use CSS Grid for but if you watch the video they never make a grid like that.

When I look at the thumbnail I see 2 columns.

In the left column I see an upper half and a lower half.

The lower half is split into 2 columns itself

The left of those 2 columns is also split into an upper and lower part.

On the right I see a 3 row layout which each row being 1/3rd of the height of the entire thing.

So, can we really make that layout with CSS Grid and make it responsive? I suppose the answer is that we can but I doubt the way we have to do it is really what CSS grid people want. Remember CSS Grid is just that, a grid, so in order to make the layout above we’d need a grid like this

That hardly seems reasonable. The left column split into 2 50% height parts really shouldn’t care about the right split into three 33% parts. The left top half really shouldn’t have to care the the left bottom half needs to split into 2. And it all gets more messed up by the left most bottom area being split horizontally. Everything gets conflated this way. If you want to add another split somewhere you might have to redo a ton of stuff. That’s not a reasonable way to go about this.

The more obvious way to do this is to nest grids just like we used to nest tables. In other words what we want is this

That’s 5 grids. The outer one with 2 sides. The inner right one with three 33% rows. The left with two 50% rows and so on.

The problem is now we lose the ability to place things arbitrarily because grids only affect their direct children.

It seems like we need yet another new css type. Let’s call it css layout. I have no idea how that would work. The problem comes back to separating content from style which means in order to do this we’d need some way to specify a hierarchy in CSS since that hierarchy shouldn’t be expressed in HTML.

I have no idea what that expression would be, some large json looking like structure in CSS like this?

.mylayout {
  display: layout;
  layout-spec: "{
    rows: [
      { height: 100%;
        columns: [
          { width: 50%;
            name: left-column;
            rows: [
              { height: 50%;
                name: top-left;
              }
              { height: 50%;
                name: bottom-left;
                columns: [
                  { width: 50%;
                    name: bottom-left-left;
                    rows: [
                       { height: auto;
                         name: bottom-left-left-top;
                       }
                       { height: auto;
                         name: bottom-left-left-bottom;
                       }
                    ];
                  }
                  { width: 50%;
                    name: bottom-left-right;
                  }
                ];
              }
            ];
          }
          { width: 50%;
            name: right-column;
            rows: [
              { height: 33%; name: right-top; }
              { height: 33%; name: right-middle; }
              { height: 33%; name: right-bottom; }
            ];
          }
        ]
      }       
    ]
  }";
}

It seems arguably what’s needed though. Unless I’m missing something CSS Grid really doesn’t solve all the layout issues what we’ve been trying to solve since nested table days.

Visual Studio Code Wishlist

I’m in the process of switching to or at least trying out Visual Studio Code.

I’m not familiar with every editor but I certainly know that an expandable editor is not a new thing. Emacs is as old as dirt and has been customizable since it was first created. vi has been customizable just as long I’m guessing. Brief, a DOS based editor from the late 80s was probably the first editor I customized a bunch with custom scripts.

For whatever reason though Visual Studio Code feels like it’s done something new. I think the biggest thing it did (and I get that I might have gotten this from Atom) is not just to have an extension system but to actually integrate an extension browser/market directly in the product. Most other extensible editors require digging deeply into the manual to find out how to add an extension and then you’re left on your own to find them. VSCode on the other hand asks you when you start or you can just click one of the main icons.

If that was all it brought that might be enough but it seems to have brought a few other innovations (expect to get corrected in the comments). One is that Microsoft designed an asynchronous language server protocol for heavy processes to run outside the editor. So for example if you want to have a C++ linter that runs LLVM to find places to warn you you don’t have to put it in the editor and have it slow down your editing. Rather you run it as an external process, it can take as long as it needs and provide the result asynchronously.

Some editors did stuff like that but they didn’t define a standard for it, instead it was just built in to their few custom linters. VSCode on the other hand, because it defined a standard opened that up to add support for any language. Write the linter, interface using the protocol, install, BOOM! Your editor now supports a new language. They even suggested other editors consider supporting the same protocol so they can take advantage of the same linters.

Maybe other editors support a similar feature but I was kind of blown away at some of the lint warnings I saw. For example I recently started playing with React. I followed the docs but of course things change over time. I followed some instructions that said to install the React linter into VSCode and suddently I was getting super React specific linter warnings directly in my editor. Is that a common thing anywhere else? I’ve certainly seen lint warnings for the language itself but I’ve never seen them for specific libraries.

It also helps, at least for JavaScript, that there’s is almost a defacto standard for declaring your dependencies. This means the editor can easily look them up and know which linters to run. Of course you can also manually specify them but dang, that’s pretty awesome.

Another feature I hadn’t seen so much before is AST based linters that can expose correctors. I’ve seen a linter tell me “2 spaces required here” or “must use double quotes” but I hadn’t seen “Fix all occurrences of this issue” before. Of course I’ve seen re-formatters before but they were generally all or nothing. You run your entire file through, it reformats everything, you get stuff out the other end. Some of you might like that kind of stuff but I’m not a fan as I find I generally want more flexibility than most coding standards provide.

This is not VSCode related but that brings up yet another thing which is the most common linter for JavaScript is eslint and not only is it massively configurable but it also takes plugins. That means unlike most linters I’m familiar with eslint you can relatively easily customize it for your personal style instead of just having to pick one of a few presets. Similarly the way these are specified are someone standardized so VSCode can adapt to difference lint settings per project.

Unfortunately VSCode is still missing many things I’m used to. Some might be easy to add as plugins. Others will require big changes I can only pray the team will make

Big needs include recordable keyboard macros. That’s a feature I’ve used often for over 30 years and it’s a little hard to believe that it wasn’t part of the MVP for VSCode. It even exists in Visual Studio. It requires a certain level of architecture at a deep level. Hopefully that architecture is already in place.

Another personal big need is support for unlimited window splitting. As it is VSCode is hardcoded to 3 vertical splits period. No horizontal splits. No 2×2. Emacs since the 80s has had unlimited spitting. In this age of 30inch monitors a limit of 3 splits, and only vertical splits, seems very limiting.

Yet another is support for column editing. VSCode has support for multiple cursors and that’s great but it’s not a substitute for column editing.

column editing

multi-cursor editing

Similarly in my old editor there are at least 3 modes of selecting. (1) Normal character to character select. (2) line select (3) column select. You press Alt-M (or just hold shift) for the normal character to character select, Alt-L switches to line select, Alt-C to column select. Moving the cursor keys once in selection mode adjusts the selection. Pressing almost anything else stops selecting which basically means you need the editor to tell you some other key was pressed. Just assigning a function to get called on Alt-L, Alt-M, Alt-C is not enough.

Mapping the keys might be easy but handling virtual space not so much.

Those things probably need core changes to the editor’s internals. The following can probably just be done with extensions some of which might already exist.

Less mouse dependence. I could be I’m just not used to it yet but in my previous editor there is an optional custom open dialog that doesn’t need a mouse. The default is to use the OS’s open dialog but when editing I find that a PITA. Another is a buffer selector. Some people might see this as the same as file selector but for some reason I find it useful to be able to switch among the short list of files I have open rather than the entire list of files in my project. Plugins for those probably would not be that hard.

Another is to be able to search and replace based on coloring or the AST. Example: search and replace “for” with “four” but only in strings. Without that feature every for loop would get replaced. Or search and replace “which” with “witch” but only in identifiers.

More auto formatting would be nice. It’s possible it already exists and I just have to configure or turn it on. As a higher level example my old editor could auto wrap comments

And it’s also aware when I’m in a comment block and knows to indent the next line with a comment. It’s probably just a minor tweak to the current auto indent which looks at the previous line.

This one might require callback/event support but my previous editor keeps backups and I can visit and diff them

You probably thinking I should use git for that but that’s not really the point. The point is this just happens in the background and saves me from when I haven’t committed to git yet or when I’m working on a file not related to software dev.

One more is safe multi-file search and replace. Basically my current editor can search and replace across files and then undo the entire thing. Of course without that feature I could manually backup first, then do my search and replace but it’s so much nicer just to be able to do it without thinking about it.

I’m not entirely sure I want to switch but I do find some of the new features compelling so I’m hoping I can get VSCode customized to surpass my previous editor.

Does Chrome need to change it’s mic/camera permission UX?

I’m a little worried about webcam and mic access in browsers at least as currently implemented. (and native apps too I suppose but less so)

As it is, in Chrome, if you let a webpage access your camera (or mic) that page’s domain gets permanent permission to access the camera or mic whenever it wants forever, no questions asked again.

I recently visited one of the early HTML5 Webcam demos that I hadn’t visited in years. I expected to get asked for permission to access the camera. Instead the camera just came on. That does not seem like a good model for web sites that are covered in ads and scripts from all over the net.

I’m sure the Chromium team was thinking of supporting hangouts when designing webcam support and I might be convinced that if hangouts always had access to the camera that might be no worse than a native app. But, it’s the browser it’s not a native app, it’s untrusted code.

Even for communications sites though if I run them in an iframe they get camera permission. In other words, say yes just once and now that domain can follow you all over the net and use your camera and mic, at least as of Chrome 59. Did you ever use your mic on facebook.com to make a call? Well now any page that has facebook social embeds can now use your mic without asking.

I don’t know what the best permission UX is. Always asking might might be tedious for actual communication websites (messenger, hangouts, slack?, …) but not asking sucks for the open web. It even sucks on a communications website if the camera or mic is not something I use often. I don’t want any app to have permission to spy on me. Imagine slack.com, imagine you use it to do a video call just once, then not again for 6 months even though you’re text chatting constantly. During that entire time, at any time slack could have been accessing your mic or you camera. I personally want to opt-in to always ask. I think I’d prefer this even in native apps but especially for the web. The UX doesn’t have to suck. Clicking “call” and having the browser say “this app wants to access your mic, Y/N” doesn’t seem like a burden.

Here’s a demo of the issue.

At a minimum it seems like iframes should not get automatic permission even if that domain had permission before. Otherwise some ad company will make a compelling camera demo just to get you to say yes once to using the camera on their domain. Once they do all their ads all over the net can start spying on you or using the mic to track you.

Even then though there are plenty of sites that allow users to post JavaScript, give access to one user’s page and all users’ pages get access. So nice person makes camera demo, bad person takes advantage that that domain now has access to your mic or camera.

I filed a bug on this about 5 months ago but no word yet. If you think this is an important issue consider starring the bug. Until then, if this concerns you go to chrome://settings/content/camera in Chrome and remove all the sites you’ve enabled the camera for. Do the same for the microphone by going to chrome://settings/content/microphone.

If you use a site that needs access to the camera or the mic, once you’ve given it permission to use the camera or the mic a small icon will appear in the URL bar. When you’re done with the site you can click that icon to remove permission. That’s tedious and you’re likely to forget but it’s better than nothing for now.

Dear Apple, Please make your app stores more internationally friendly

I know we are in the minority but as a person living abroad (Japan) it’s extremely frustrating to use the iOS (and Mac) App stores.

Because my native language is English and I’m from the USA I want to use the USA store. But, sometimes I have to use the Japanese store, for example to download a banking for local banks who’s apps are only available on the Japanese store. Doing this is extremely tedious. The steps are:

1. Go to your home page, swipe to whatever page your settings are on and click the settings app.

2. Scroll down, find, and pick “iTunes and App Store”

3. Click your Apple ID (it’s not clear at all this is clickable)

4. Pick “View Apple ID” (It’s not clear at all this is where you’d change the store)

5. Enter you password. I have a long password with both numbers and symbols. It’s super annoying to type on the phone. No idea why this is Japanese right now.

6. Pick “Country/Region”.

7. Pick “Change Country or Region” (think of this as “yes, I really want to do this”).

Imagine the frustration of students etc who don’t have a local credit card with which to sign up. I’m guessing they beg their friends.

8. Select the region you want.

9. Enter your local address and postal code.

10. Enter your local phone number

11. Enter a local credit card, its expiration date and its code.

All of this takes several minutes.

Then, when I need to switch back I have to go through the same tedious process.

To give an example, Saturday I wanted to checkout Japanese podcasts. The easiest way to access the most popular ones seemed to be to use the builtin Apple iOS Podcast app. So I click it but of course since I’m on the USA store it shows the USA list. I sigh and decide I’ll do this later. A couple of days later I finally get around to switching stores. I check the podcasts just to see it’s showing me the Japanese lists. It is but don’t download any yet. Today I need some network info app for iOS. I find one called “Network Utility” on iOS and download it. It’s not showing the info I need. I check their website to see if there’s an example of it showing the info I need. It’s possible the info doesn’t exist on my phone. I see the same app exists on the Mac App Store. I check it out, I can’t tell but I know the info I want to display on iOS *is* available on my Mac. In other words if the Mac App displays the info I want then maybe the iOS one does too. There’s no free version but it’s only 99¢ so I click “buy” and am then told I can’t use the account to buy it. In other words because I switched my region to Japan and this mac app is on the USA store I can’t buy it without going through all the shit above to switch back to the USA region.

Now I get I’m in the minority, most people stay in whatever region they were born in so this issue probably affects few people (maybe tourists?). But at the same time why does it have to be this way? Why can’t I just register addresses and payment info for both stores and switch easily without having to re-enter it all every time? Or better why I can’t have it automatically use the correct info. Or even better than that why can’t there just be one store? Let me select a region for the charts and recommendations but otherwise put it all on the same store! Come on Apple! You’re the leaders in UI/UX. You can do this!

Don’t disable web security!!!

This basic question is all over stack overflow.

People ask how can they access files when developing HTML locally. They make a .HTML file, then open it in Chrome. They add a script that needs to access an image for canvas or WebGL or whatever and find they can’t. So they ask on Stack Overflow and the most common answer is some form of “Start Chrome with the option –disasble-web-security” (or one of 5 or 6 other similar flags)

I keep screaming DON’T DO THAT! but almost no one listens. In fact not only that the downvote my answers.

Well here’s two proof of concepts of why it’s ill-advised to disable web security.

The first one is an example that will get your stack overflow or github username if you are logged in and you started chrome with --disable-web-security. Of course you probably don’t care that someone is looking up your username on various sites but that’s not really the point. The point is some webpage not related to those sites was able to access data from those sites. The same webpage could access any other site. You bank, your google account, all because you disabled security.

You might say “I’d never run a script like that” but you likely run lots of 3rdparty scripts.

The second example will show files from your hard drive. It could upload them to a remote server. Which files some baddie would want I have no idea. The point is not to show uploading dangerous files. The point is only to show if you disable web security it’s possible for a script, your own or a 3rd party one to access your local files.

Many of you will be thinking “I’d never do either of those” but I think that’s being short sighted. I know I often forget which browser I’m in, the dev one or the non-dev one. If I mistakenly used the dev one with web security disabled then oops.

Of course you might also be thinking you’d never do any of the things above. You’re running your own hand coded webpages with scripts and not using any 3rd party libraries and you never use the wrong browser. But again, that’s not the point. The point is you turned off security. The point is not to enumerate all the ways you might get hacked or have data stolen or accounts manipulated. The point is if you disable web security you’ve made yourself more vulnerable period.

This is especially frustrating because the better solution is so simple. Just run a simple local server! It will take you all of 2 minutes at most. Here’s one I wrote for those people not comfortable with the command line. Here’s also 6 or 7 others.

Sony Playlink

So Sony announced Playlink at E3 this year 2017.

It’s a little frustrating. It’s basically HappyFunTimes.

The part that’s frustrating is in that video Shuhei Yoshida is happily playing a Playlink game and yet about a year ago I saw Shuhei Yoshida at a Bitsummit 2016 party and I showed him a video of happyfuntimes. He told me using phones as a controller was a stupid idea. Is objection was that phone controls are too mushy and laggy. He didn’t have the foresight to see that not all games need precise controls to be fun. And yet here he is a year later playing Playlink which is the same thing as happyfuntimes.

In 2014 I also showed Konou Tsutomu happyfuntimes at a party and even suggested a “Playstation Party” channel with games for more than 4 players using the system. My hope was maybe he’d get excited about it and bring it up at Sony but he seemed uninterested.

Some people don’t see the similarity but I’d like to point out that there are

  • There are happyfuntimes games where you draw on the phone
  • There are happyfuntimes games where you use a virtual dpad
  • There are happyfuntimes games where you tilt and orient the phone like a Wiimote
  • There are happyfuntimes games where you set the phone on the table and spin it
  • There are happyfuntimes games where you’re given various menus of choices
  • There are happyfuntimes games that take live video from the phone and display it in the game
  • There are happyfuntimes games that use the phone like Google Cardboard, multiple people looking in shared VR
  • There are happyfuntimes games that play sounds from the phone
  • There are happyfuntimes games with asymmetric controls where
    • One player is the driver (uses phone like Wii MarioKart Wiimote)
    • One player is the navigator (map only on his phone so has to direct driver)
    • Two players are chefs making pizzas, pretending phones are boxes of ingredients

And many others.

And of course there are also games you could not easily play on PS4 like this giant game where players control bunnies or this game using 6 screens.

And to Shuhei’s objection that the controls are not precise enough

You just have to design the right games. Happyfuntimes would not be any good for Street Fighter but that doesn’t mean there aren’t still an infinite variety of games it would be good for.

In any case I’m not upset about it. I doubt that Shuhei had much to do with Playlink directly I doubt Konou even brought it up at Sony. I think the basic idea is actually a pretty obvious idea. Mostly it just reinforces something I already knew. That pitching and negotiation skills are incredibly important. If I had really wanted happyfuntimes to become PS4 Playlink I should have pitched it much harder and more officially than showing it casually at a party to gauge interest. It’s only slightly annoying to have been shot down by the same guy that announced their own version of the same thing. 😜

In any case, if you want to experiment with games that support lots of players happyfuntimes is still a free and open source project available for Unity and/or HTML5/Electron. I’d love to see and play your games!

Wishing for more Sandboxes

I’m starting to wish that nearly all desktop apps ran in a very tight sandbox the same way they do on iOS.

Windows is trying to do this with the Windows store and Apple is trying to do it with the Mac App Store. The problem is two folder. One is they started with unsandboxed systems and so have decades of legacy software that expects to be unsandboxed. The other is they’ve conflated sandboxes and their app stores. Those 2 things should be separated.

Apps like Photoshop, Lightroom, Microsoft Word, gIMP, Blender, Maya, etc should not need system wide access.

To be clear I am **NOT** suggesting that there should be an app store or there should be an approval process for apps. Rather I’m suggesting that the OS should default to running each app in a sandbox with that app unable to get outside its sandbox without user permission. The permission system should be designed well (like I think it mostly is on iOS) so a native app should not be able to access your entire hard drive by default. It should not be able to read files from other apps by default. It should not be able to use your camera or mic or get GPS info by default. It should not be able to supply notifications by default or read your contacts. All of those things should be requested of the user at use time like iOS does (and I think Android is in the process of doing).

This might seem unrelated but it came up recently when a user on Stack Overflow asked how to make an Electron app from their HTML5 WebGL game. There are a few steps but over all it’s pretty easy. If you’re not familiar with Electron it’s basically a version of Chrome that you can bundle as an app with your own HTML/CSS/JavaScript but unlike a normal webpage your JavaScript can access native features like files, OS level menus, OS level networking, etc.

And there in is the issue. The issue is it’s common to use 3rd party scripts in your HTML5 apps. Maybe you’re including JQuery or Three.js from a CDN. Maybe like many mobile apps you’re downloading your HTML/CSS/JavaScript from your own servers like myautoupdatingapp.com. By doing that you’ve just made it possible for the people controlling the CDN or that hacks your server or the people that buy your domain to own every machine that’s running your app. This is something that’s not true with a browser doing the same thing because the browser does not allow JavaScript to access all those native things. It’s only Electron that does this.

This means I have to trust every developer using Electron to not do either of those things.

On the other hand, this is exactly what iOS was designed to handle. You don’t have to trust the app to the same level because the OS doesn’t let the app read and write files to the entire machine. The OS doesn’t let the app access the camera or the mic without first asking the user for permission.

This isn’t the first time this kind of thing has happened. I’m sure there’s plenty of other cases. One for me is XBMC/Kodi where there are plugins but no sandbox which means every plugin could be hacking your system. Many of those plugins are for websites that are arguably doing questionable things so why should I trust them not to do questionable things to my machine?

I’d even take it so far as I wish it it was easier to do this in the terminal/shell. If I’m trying out a new project there is often a build step or setup step or even the project itself. Those steps often allow code to run, code I don’t want to have to trust. Of course in those cases I could run them in a VM and maybe I should start doing that more. I’m just wishing that that was easier than it is today. Like it kind if wish it was an OS level thing. I’d type something like

mkdir test & cd test & start VM

or

mkdir test & cd test & start standbox

Then I could

git clone someproject .
./configure
make

or

git clone somejsproj .
npm install

And not have to trust the 1000+ contributors above that they weren’t doing something bad intentionally or unintentionally.

Unfortunately without a push by Apple and/or Microsoft it’s unlikely the big software companies like Adobe are going to switch to their apps to the sandboxed systems.

IMO both companies need to separate their sandboxes (good) from their stores (bad). They then need to make it harder to run un-sandboxed apps. Not impossible, some apps probably need system level access if they provide system level services. But, they need to start making it the normal that the apps themselves are sandboxed.

NES/Famicom, A Visual Compendium – Corrections

I very nice book came out with images from tons of NES games called “NES/Famicom, A Visual Compendium“. I saw several of my friends had backed the kickstarter and when they go their copies they mentioned one of my games was in it so I had to buy a copy.

It’s a gorgeous book. I’m not 100% sure I’m into the sharp emulator captures graphics as they look absolutely nothing at all like the original games. As art they’re very cool but as representations of what those games looked like they are far off the mark. You can see a comparison on this article and see how when blown up in an emulator they look all blocky but back when the came out they looked smooth. Still as graphic art it’s cool to see them in the book.

That said I looked up M.C. Kids and was a little disappointed to see things reported incorrectly. I not blaming anyone in particular. I assume it’s an issue of like the game “telephone” where as the message got passed from person to person it got re-interpreted and ended up in it’s present form

For M.C Kids it implies Rene did all the enemies but that was not the case. I’m not sure the percent of enemies created by Rene but IIRC Darren Bartlett and Ron Miller both did enemies as well.

I then notice there was an unreleased section and sure enough there was Robocop vs Terminator listed.

It’s also wrong. It’s says “Graeme Devine moved me from Caesars Palace Gameboy to Robocop Vs Terminator”. What actually happened though is Graeme Devine moved me from Caesars Palace to Terminator NES, not Robocop vs Terminator. I worked on an animation tool to be shared between Terminator NES and M.C. Kids NES. Later I was asked to work on M.C Kids and Terminator was given to David Parry to make Terminator for Sega Genesis. When I finished M.C. Kids and I was no longer at Virgin Games I got a contract from Interplay to code Robocop Vs Terminator for NES.

Trying to help noobs is SOOOO FRUSTRATING!

I often wonder if I’d like to teach. It’s certainly fun to teach when the students are easy to teach ? But where’s the challenge in that?

I wrote webglfundamentals.org (and webgl2fundamentals.org) and I answer tons of WebGL questions on stackoverflow but sometimes it’s sooooooooooooooo frustrating.

I’m trying to take those frustrations as an opportunity to learn better how to teach, how to present things, how to be patient, etc but still… (more…)