Making an App – Part 3

Let's put some markers on the map.

Fortunately we can get marker data from the same place we got maps tiles. The following command will download some JSON that describes the markers to botw_markers.json:

$ curl "https://zeldamaps.com/ajax.php?command=get_markers&game=19" > botw_markers.json

We should trim it down a little since there's too much data. It's a little easier to see if we reformat it with jq which will "pretty print" it by default:

$ cat botw_markers.json | jq '.' | less

Each marker looks like this:

{
  "id": "8549",
  "mapId": "19",
  "submapId": "1901",
  "overlayId": null,
  "markerCategoryId": "1905",
  "markerCategoryTypeId": "1",
  "userId": "1",
  "userName": "|N|NjA|",
  "name": "Pot Lid",
  "description": "GameID: 871017963",
  "x": "155.90646875",
  "y": "-152.331625",
  "jumpMakerId": "0",
  "tabId": "",
  "tabTitle": "",
  "tabText": "",
  "tabUserId": "",
  "tabUserName": "",
  "globalMarker": "0",
  "visible": "1"
}

It would be better if we trimmed it down to something like this:

{
  "category": "1905",
  "name": "Pot Lid",
  "description": "",
  "x": "155.90646875",
  "y": "-152.331625"
}

That's relatively easy with jq – map the elements of the array into a new array of elements without "pretty printing" into markers.json:

$ cat markers.json | jq -c '[.[] | {category: .markerCategoryId, name: .name, x: .x, y: .x }]' > markers.json

There's some interesting text in the "tabText" field, but for now we just want to show the markers on the map.

The "category" field will map to a category which we can also get the JSON for:

$ curl "https://zeldamaps.com/ajax.php?command=get_categories&game=19" > botw_categories.json

Like with markers, let's format the data a little:

$ cat botw_categories.json | jq '.[0]'

A category looks like this:

{
  "id": "1931",
  "parentId": "1930",
  "name": "Enemy Camp",
  "checked": "1",
  "img": "BotW_Enemy-Camp",
  "color": "#ff422e",
  "markerCategoryTypeId": "1",
  "visibleZoom": "5"
}

One thing that jumps out is that there's a heirarchy and a category may have a parent. We'll trim it down just a little:

$ cat botw_categories.json | jq -c '[.[] | { id: .id, parent: .parentId, name: .name, color: .color }]' > categories.json

Next step will be parse these into Swift types which we'll do in the next post.

Making an App – Part 2

From last time, we have a set of map tiles and full map images, so let's see if we can get an app up and running showing a map.

Creating the App

With Xcode, let's create a single view, universal iOS app, call it BotW, using the Swift language, and save it in the botw folder we created last time.

In the main storyboard, add a UIScrollView as a child of the root view and add constraints for top, bottom, leading, and trailing to the superview. If it's indented on the left and right, make sure "Relative to Margin" isn't selected.

Add a UIImageView to the scroll view and constrain it the same way.

To keep things easy, let's use one of the assembled maps, not tiles; level 4 is 3072x2560 and 7.7Mb which is a good size to start with and enough detail to be useful. Drop 4.png into the asset catalog and set the image property of the image view to "4".

That should actually be enough to get a minimally working app. Let's set the simulated device to an iPad Air and run it.

Getting Zooming Working

You can scroll around, but not zoom. That's because the scrollview needs a class that is a delegate that implements UIScrollViewDelegate, specifically the viewForZooming(in:) method which returns which view should be zoomed. That, of course, is the image view, so let's:

  1. Add an outlet for the image view to the view controller.
  2. Wire up the image view to the outlet on the view controller in the storyboard (admission – I always forget to do this at least once in every project!).
  3. Change the minimum and maximum zoom of the scroll view to 0.25 for minimum and 4 for maximum.
  4. Implement the delegate protocol in an extension of the view controller.
  5. Implement the zoom method to return the image view.
  6. Wire up the delegate of the scroll view to the view controller. If your scrollview still won't zoom when you run the app, it's probably because you forgot to do this.

Here's what the view controller looks like:

import UIKit

class ViewController: UIViewController {

    @IBOutlet var imageView: UIImageView?

    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view, typically from a nib.
    }
}

extension ViewController: UIScrollViewDelegate {
    func viewForZooming(in scrollView: UIScrollView) -> UIView? {
        return imageView
    }
}

Build & run … looks good. Let's commit the changes.

Adding State Restoration

This is a minor thing, but it's easy and makes the app better – let's have the app remember the map configuration between launches. To do this, we add a storyboard ID to the view controller and check "Use Storyboard ID" for the restoration ID. Then we add restoration ID's for the root view, scroll view, and image view. Lastly, we add the following callbacks to the app delegate:

func application(_ application: UIApplication, shouldSaveApplicationState coder: NSCoder) -> Bool {
    return true;
}

func application(_ application: UIApplication, shouldRestoreApplicationState coder: NSCoder) -> Bool {
    return true;
}

To test it, build & run, zoom & scroll to some point, hit the "home button" on the simulator with ⇧⌘H and then stop the app in Xcode. Now, the next time you launch the app the map should return to that same zoom & scroll point.

Commit changes.

Centering the Zoomed Out Image

Two minor annoyances. First, when you zoom out the image anchors to the top left. It would be nicer if it was centered.

While iOS 11 and Xcode 9 have an easy fix for this, the current version needs some more help. Basically, what we want to do is adjust the constraints on the image view to inset enough to center the image when either the height or width is less than the scroll view size. So, compute the difference, divide by 2, and add as constants to the constraints. This means we need references to the constraints, so first let's add outlets and wire them up.

Now, let's adjust the constants when the scroll view zoom changes in the delegate protocol extension.

func scrollViewDidZoom(_ scrollView: UIScrollView) {
    if let view = self.imageView {
        let widthOffset = max(0, (scrollView.frame.width - view.frame.width) / 2)
        let heightOffset = max(0, (scrollView.frame.height - view.frame.height) / 2)

        imageViewTopConstraint?.constant = heightOffset
        imageViewBottomConstraint?.constant = heightOffset

        imageViewLeadingConstraint?.constant = widthOffset
        imageViewTrailingConstraint?.constant = widthOffset
    }
}

That works … until you rotate the device. The constraints need to update on device rotation, too. The view controller will get a callback to viewWillTransition(to:with:) and will get a CGSize which is all we really need anyway.

First, commit changes.

Now, let's pull this code out into a function on the controller:

func adjustContraints(for size: CGSize) {
    if let view = self.imageView {
        let widthOffset = max(0, (size.width - view.frame.width) / 2)
        let heightOffset = max(0, (size.height - view.frame.height) / 2)

        imageViewTopConstraint?.constant = heightOffset
        imageViewBottomConstraint?.constant = heightOffset

        imageViewLeadingConstraint?.constant = widthOffset
        imageViewTrailingConstraint?.constant = widthOffset
    }
}

Now we can update the delegate protocol implementation to call it:

func scrollViewDidZoom(_ scrollView: UIScrollView) {
    adjustContraints(for: scrollView.frame.size)
}

And do the same on the size transition callback in the view controller:

override func viewWillTransition(to size: CGSize, with coordinator: UIViewControllerTransitionCoordinator) {
        adjustContraints(for: size)
}

Build & run … working well. Commit changes.

Commit & Take a Break

Good stopping point for now and a good point to commit the code if you haven't been all along. A habit I try to follow is to make one significant change at a time and commit the code as soon as it works. This gives me reliable "rollback" points if a refactor goes bad, or a new feature implementation goes bad. Avoid the temptation to change lots of things at once. Even large changes that touch lots of classes in a big project are good to break up into several small changes; just make sure the app builds & runs (& tests pass if you have them) before each commit.

Why So … Plain?

Some might be wondering why not just use some Cocoapod or Carthage package for tiled, scrolled images, like ARTiledImageView or why we don't do some new functional reactive approach with something like ReactNative. Why are we using storyboards (aren't those supposed be bad)?

First, don't fight the framework. I've had to learn this lesson the hard way so many times, it's drilled into me now. Whenever you choose to go in a different direction from where the framework you are working in is going, you're signing up for extra work now and especially in the future. ReactNative or RxSwift or combinator parsers may get lots of blog posts and seem hopelessly cool, but understand that they are based on opinions that are orthogonal to UIKit or AppKit.

That doesn't make them bad, but it does make them more work. When something doesn't work, is it your bug, are you using the new framework wrong, does it have a bug, or are you not understanding some underlying UIKit issue that's being obscured by the new framework? Long and short, don't sign up to make things harder just for the fun of it. If you are intrigued with one of those new things, by all means learn them. But, and I've seen this mistake so, so many times (and even made it myself), don't use a new project to learn a new technology or approach. Don't try a new cake recipe for the first time for your daughters next birthday party.

Second, don't assume your project is special until it is. Lots of people will tell you how terrible storyboards and interface builder are or how crappy Xcode is or how slow or inelegant or whatever some part of UIKit or Foundation is. And maybe for them and their special snowflake project, that's all true. However, don't volunteer to roll your own thing or avoid features of the platform that clearly have strategic importance to the platform vendor. If Apple is supporting storyboards and making strategic investments in it, there's a reason. They don't get everything right and there will be bugs, but deciding you're too cool for things like storyboards before you actually are is just masochism. Don't decide to raise your own chickens because the eggs from the store aren't fresh enough to make your cake until, you know, they actually aren't fresh enough for you. And stop listening to the prophets of negativity on the Internet; they aren't your friends.

Lastly, be wary of "accidental complexity". For any engineering project, there is something called essential complexity which is the fundamental nature of the problem itself, the actual thing you need to solve. There will be some additional complexity that gets added to the effort that has nothing to do with the actual problem, but comes along because of the approach, the tools, or the platform (e.g. like code signing). This is called accidental complexity. Every extra thing you add to the project increases accidental complexity and while some of it is unavoidable, you should try to keep it to a bare minimum.

So, since this project is all about getting a fast, working map with thousands of markers on my daughter's iPad, I'm not playing around with experimental non-Apple frameworks, rolling my own UI frameworks or layout techniques, or trying out other people's solutions to similar problems. Until I need to.

Making an App – Part 1

I've been playing Zelda: Breath of the Wild with my daughter recently. Since she's kind of a "completionist" and wants to complete all the hundreds of shrines, quests, and extras in the game, I went looking for a decent map with locations called out.

We found Zelda Maps which is decent and even installs as an app on an iPad. Unfortunately, my daughter's iPad is one of the original iPad mini models, which is basically an iPad 2 in a smaller case, and the site really just doesn't run well enough to be usable. Even on my iPad Air it runs OK, but still not great. The UI is geared towards a mouse and keyboard with very small controls which is also problematic on a small iPad and the site has ads and ad trackers running.

After poking around the Javascript and using Safari's web inspector, I could see how the app worked so I thought I'd use the site's data to make a native iPad app for her that would run well and maybe have some extra features to boot.

Making an App … In Public

I thought it might be interesting to post about the process of making the app, especially since it's not really destined for the AppStore, and not as a neatly packaged, finished product, but the way most apps get made – a little bit at a time and sometimes not so pretty.

Checking Out the Data

Looking inside the web site, it's built using the Javascript map library Leaflet. The map imagery is tiled with 256x256 image tiles at nine zoom levels, from level 0 which is a single, 1x1 tile set, up through level 8, which is a massive 188x160 tile set of 48k x 40k pixel image.

The markers are pretty easy, too, since they in basically two JSON feeds, one for all of the individual markers and another for the categories of markers.

That's nearly 1gb of map data and over 3,000 map markers … no wonder the web app doesn't run so well on an iPad mini.

Showing a large, zoomable and scrollable image with overlays is well within the kind of thing iOS and UIKit can do well, so this shouldn't be all that hard.

Getting the Data

Considering we have nearly 50,000 files to download and some images and JSON data manipulate, we've got some coding to do. You might be tempted to write some Python, Ruby, Perl, or even Swift to do it, but I always try to stick with shell scripts for this kind of thing where I can – Bash is very capable and there some excellent command line tools for image and JSON manipulation.

I recommend curl, ImageMagick, and jq. If you don't have them, install Homebrew and then install them.

When I need to run some commands several times, I like to write a script. First, let's setup a project directory:

$ mkdir botw
$ cd botw
$ mkdir tiles
$ mkdir scripts
$ mate scripts/download

I tend to use Textmate for editing scripts and other non-Xcode file, but vi works fine, too (or any other text editor you prefer). Start with a shebang and assume we're getting a zoom level, a min & max column, and a min & max row. Experience and Cocoa have taught me that calling things what they are with minimal abbreviation is always worth the effort.

#!/bin/bash

scale=$1
col_min=$2
col_max=$3
row_min=$4
row_max=$5

width=$(( col_max-col_min+1 ))
height=$(( row_max-row_min+1 ))

We'll need to know the width and height later, so a little arithmetic here. If this were destructive script, I'd put some validation and argument parsing in, but we're going to inch our way to a solution so not really important now.

I usually will "dry run" my scripts by just echoing the scripts first. We need to download a bunch of PNG files in this form:

(scale)_(column)_(row).png

To keep things easy, we'll name them the same. So a loop in a loop should do the trick:

for col in `seq ${col_min} ${col_max}`
do
  for row in `seq ${row_min} ${row_max}`
  do
    pattern=${scale}_${col}_${row}.png
    echo curl https://zeldamaps.com/tiles/botw/hyrule/${pattern} -o ${pattern} -s
  done
done

Save it and back in the terminal make the command executable and try it out. I happen to know that scale 1 has tiles from 0 to 1 in rows and columns, so let's try it:

$ chmod +x scripts/download
$ ./scripts/download 1 0 1 0 1

That produces this:

curl https://zeldamaps.com/tiles/botw/hyrule/1_0_0.png -o 1_0_0.png -s
curl https://zeldamaps.com/tiles/botw/hyrule/1_0_1.png -o 1_0_1.png -s
curl https://zeldamaps.com/tiles/botw/hyrule/1_1_0.png -o 1_1_0.png -s
curl https://zeldamaps.com/tiles/botw/hyrule/1_1_1.png -o 1_1_1.png -s

Let's try one of those commands out:

$ curl https://zeldamaps.com/tiles/botw/hyrule/1_0_0.png -o 1_0_0.png -s
$ identify 1_0_0.png
1_0_0.png PNG 256x256 256x256+0+0 8-bit sRGB 45494B 0.010u 0:00.009

Looks like we got a good download, so let's delete test file, remove the echo from the curl command, add some extra status messaging, and add an extra parameter for a prefix so we can direct where the files download to.

#!/bin/bash

scale=$1
col_min=$2
col_max=$3
row_min=$4
row_max=$5
prefix=$6

width=$(( col_max-col_min+1 ))
height=$(( row_max-row_min+1 ))

for col in `seq ${col_min} ${col_max}`
do
  for row in `seq ${row_min} ${row_max}`
  do
    pattern=${scale}_${col}_${row}.png
    echo -n "Downloading ${pattern} ... "
    curl https://zeldamaps.com/tiles/botw/hyrule/${pattern} -o ${prefix}${pattern} -s
    echo "OK"
  done
done

Now let's run it ...

$ ./scripts/download 1 0 1 0 1 tiles/
Downloading 1_0_0.png ... OK
Downloading 1_0_1.png ... OK
Downloading 1_1_0.png ... OK
Downloading 1_1_1.png ... OK

Of course, some of these downloads are going to be pulling down hundreds or thousands of files … would be nice if it could be started and stopped without starting all over. Let's put a check in for already downloaded files and test it out

#!/bin/bash

scale=$1
col_min=$2
col_max=$3
row_min=$4
row_max=$5
prefix=$6

width=$(( col_max-col_min+1 ))
height=$(( row_max-row_min+1 ))

for col in `seq ${col_min} ${col_max}`
do
  for row in `seq ${row_min} ${row_max}`
  do
    pattern=${scale}_${col}_${row}.png
    if [ -f ${prefix}${pattern} ]
    then
      echo "Skipping ${pattern}"
    else
      echo -n "Downloading ${pattern} ... "
      curl https://zeldamaps.com/tiles/botw/hyrule/${pattern} -o ${prefix}${pattern} -s
      echo "OK"
    fi
  done
done

Now testing ...

$ rm tiles/1_0_0.png 
$ ./scripts/download 1 0 1 0 1 tiles/
Downloading 1_0_0.png ... OK
Skipping 1_0_1.png
Skipping 1_1_0.png
Skipping 1_1_1.png

Works nicely, time to download 10,097 files (I decided to skip level 8 since it had no actual additional detail, just level 7 doubled).

Assembling the Tiles

While it's possible to display large, tiled views with multiple zoom levels using CATiledLayer, I do want to checkout that the tiles I've pulled down look good together and it might help to use a single, assembled map as a starting point so we don't have to implement tiling right away.

ImageMagick has a tool that does this easily, montage. It takes many options and can produce many kinds of output, but we just want to assemble the tiles in order to produce the original, untiled image. To do this, we simply need to pass the list of images, how much padding to use, and the output file name.

Like so, using zoom level 2 which is a 4x4 grid.

$ montage tiles/2_*.png -geometry +0+0 2.png

montage can usually figure out layout, height and width, but looking at the output it's not working right; the tiles that should be going down the left are going across the top. It's mixing the rows and columns. Turns out that montage is wanting the tiles to be sorted into rows, but our tiles are sorting by column since the column is first in the name.

Fortunately, sorting in the shell is pretty easy; we want to sort the file name by row which is the third number in the name and then by the row. A quick look at the manual page for sort shows we need to pass a separator and keys. This seems to give the correct result (the n in -k3n -k2n is important since it indicates the values are numbers so that we don't get "1, 11, 12, … 2, 21, … 3, …", but "1, 2, 3, … 11, 12, …, 21, …"):

$ ls tiles/2_* | sort -t'_' -k3n -k2n

Now we give this to montage and we get the output we want:

$ montage `ls tiles/2_* | sort -t'_' -k3n -k2n` -geometry +0+0 2.png

Unfortunately, starting at level 5 the grid isn't square so montage guesses wrong on the layout. We need to help it out. It looks like the map is 24 tiles wide at level 5 so we can hint it (and double that at 6 and 7, so 48x and 96x):

$ montage `ls tiles/5_* | sort -t'_' -k3n -k2n` -geometry +0+0 -tile 24x 5.png

These larger maps can really chew up some CPU and run for a bit so you might want to use both nice and caffeinate to lower the priority of the job and keep the power saver from kicking in while it's running.

Once that's done, that's a pretty decent stopping point. Next post will pull down and clean up the JSON data.

Running Snipposé at Login

One request I've gotten for my recently published macOS app, Snipposé, is to add an option to automatically start at login.

While I've added that to the feature request list, there's no reason to wait for me to add that to the app itself. You can run any app on login on a Mac easily.

First, launch System Preferences, open Users & Groups, and click on "Login Items". Or search for "Login Items" in Spotlight or in the System Preferences app's search field.

Click the "+" button, select "Snipposé" from "Applications", and click "Add".

That's pretty much it. From now on, when you login, Snipposé will be there waiting for you on the menu bar.

Cocoa Screen Capture with Multiple Screens

In my recent Snipposé project I ran into a bug in my code when using CGWindowListCreateImage.

I had assumed that the GCRect for the screenBounds parameter was relative to the frame of the screen of the window I was calling convertToScreen on. It is not. It's relative to the bounds of this function: CGDisplayBounds(CGMainDisplayID()).

As usual, the coordinate space is inverted from the window so you need to flip it. The following snippet shows the code to grab a screen capture using a rect relative to a window:

// "rect" is the frame to capture in the coordinates of "window"

let mainDisplayBounds = CGDisplayBounds(CGMainDisplayID())
var captureRect = NSRectToCGRect(window.convertToScreen(rect))
captureRect.origin.y = mainDisplayBounds.height - captureRect.origin.y - captureRect.height
if let image = CGWindowListCreateImage(captureRect, .optionOnScreenBelowWindow, CGWindowID(window.windowNumber), []) {
    let screenImage = NSImage(cgImage: image, size: rect.size)
    // do something with your image, now
}        
Archive - Snipposé - Game Friends - Rename Finder Items - The Fox & The Grapes