Cocoa Screen Capture with Multiple Screens

In my recent Snipposé project I ran into a bug in my code when using CGWindowListCreateImage.

I had assumed that the GCRect for the screenBounds parameter was relative to the frame of the screen of the window I was calling convertToScreen on. It is not. It's relative to the bounds of this function: CGDisplayBounds(CGMainDisplayID()).

As usual, the coordinate space is inverted from the window so you need to flip it. The following snippet shows the code to grab a screen capture using a rect relative to a window:

// "rect" is the frame to capture in the coordinates of "window"

let mainDisplayBounds = CGDisplayBounds(CGMainDisplayID())
var captureRect = NSRectToCGRect(window.convertToScreen(rect))
captureRect.origin.y = mainDisplayBounds.height - captureRect.origin.y - captureRect.height
if let image = CGWindowListCreateImage(captureRect, .optionOnScreenBelowWindow, CGWindowID(window.windowNumber), []) {
    let screenImage = NSImage(cgImage: image, size: rect.size)
    // do something with your image, now
}        

Swift Workgrounds?

Recently updated my iPad to the iOS 10 beta solely so I could check out Swift Playgrounds and all I have to say is, "Wow!"

Just about everything about it is just super impressive. Two things in particular really blow me away:

  • The editor.
  • This is actual, full-on Swift. With UIKit. Let that sink in for a minute.

The editor is brilliant. I absolutely love that it feels like a cool mix of Scratch and a text editor, the way it handles "auto complete" is nicely adapted to touch, and I kind of wish I could use the "power" keyboard everywhere.

It's decidedly not an IDE (which is fine). It's not trying to be, it's a playground and a great fit for an iPad.

But what really is making me feel a little giddy & lightheaded is that this is really Swift, not some jacked up "SwiftScript" or constrained subset. This is the real deal with access to the Cocoa Touch frameworks. The real thing. When I hit "Run My Code", there's a Swift compiler & linker producing a binary and running it. On my two year old iPad Air. I don't what the actual implementation is, but the result is effectively the same.

It makes me giddy because if we can do this in a playground, and we can obviously do it safely enough that Apple isn't afraid of it, where else will we compile & run Swift on the fly?

This may be crazy talk, but just think about this little thought experiment: if a browser can download source code (Javascript), some UI layout (HTML, CSS), and some resources and run them in a rectangle to present a useful app (sometimes slowly & painfully), what's stopping it from downloading a different set of source code (Swift) and UI layout (storyboards & xibs) and running those in a rectangle to present a useful app?

Web apps have basically turned into the defacto platform for workaday and line-of-business apps. As mobile becomes the dominate computing platform, having something better than the somewhat creaky web stack could be super appealing.

Privacy is this Century's Tobacco

Absolutely brilliant and prescient.

From the podcast Is the shine off (the) Apple? episode at the 15:53 mark:

Horace: "I think you're right, I think consumers don't care [about privacy], and indeed they didn't care about tobacco killing them and they didn't care about sugar killing them and they didn't care about fascism and a lot of other destructive forces in society ..."

Host: "It just got very real, Horace!"

Horace: "Well, I don't really think it's at all different. Privacy is more important than a lot of these health issues. It is going to be the 'Tobacco' of the 21st century without a doubt. And whoever's on the wrong side of that bet is going to get completely decimated. I mean it's just going to be tragic."

As the Jacobite prince is fond of saying on the Outlander series, "Mark me," I think this worm will turn. And when it does, it may be a rough time for Facebook, Google, and other social sites that depend on people donating or selling their personal data.

The Swift Safety Distraction

I keep thinking about why I feel uneasy about the current fad of static typing and functional programming.

I mean, I don't have anything against either. Well, maybe a little bit against static typing but only because it can be a real nuisance sometimes. But I've been interested functional programming ever since I first ran into Erlang nearly 10 years ago. I've been trying to carry along ideas I've picked up from other languages and frameworks into every new language, platform, and framework I pick up.

I do like Swift; I think it's one of the most interesting and practical languages I use today. I sometimes feel like people want me to despise Objective-C, now, which I can't do. I consider it one of the most interesting languages with some of the best frameworks on some of the coolest platforms I've ever had the joy to work with.

I think what's bugging me is the focus on "safety"; this idea that Swift is great because it will prevent us from writing "bad code" and force us to write good code. I can't shake the feeling this is a useless and maybe even dangerous distraction. I'm not certain, I could easily be wrong here, but I want to get this funny feeling off my chest, to talk it out.

I feel Swift is an improvement because it gives us more power with less effort. If the compiler can catch a couple extra errors for me, that's a bonus, but that alone doesn't seem to be all that special.

To illustrate, let's take generics. They most often seem to be presented as tools to prevent mistakes. Well, I feel generics are best when they are used like a lever, not a gate. A good generic type design can make code more expressive and more dense – less cruft. But that takes mostly design effort, not necessarily a language feature.

Take generic collections; we're mostly told they're great because they prevent us from adding the wrong type to a collection.

var strings:[String] = []
strings.append(42) // no, no, no! not it my house!

Um, OK, I guess. But, I don't really need you to protect me from myself. In 25 years of professional programming, I don't think I've ever had a problem adding the wrong type to a collection. Not once.

Now, I've gotten collections that had mixed types in them, like parsed JSON, and I've screwed up handling them (or inherited code that did), but generics don't help me here. Why? Because the core problem remains. If the code I'm calling is giving me collections with mixed types, making it use generics isn't going to fix a bad design. In fact, the code will probably just declare the return type as [Any] and call it a day (like JSON). Now I'm stuck worrying about the same problem and the static nature of the language is just going to require more ceremony. I'm probably going to end up with lots of checking and casting code and I might still get errors at runtime.

Granted, Swift's optionals can really help here, but the advantage has nothing to do with generics or type safety. It's the introduction of an optional type, a change in the semantics of nil. Take this contrived JSON example in Objective-C:

NSString *json = @"{\"name\": \"sam\", \"height\": { \"feet\": 5, \"inches\": 8 } }";
NSData *data = [json dataUsingEncoding:NSUTF8StringEncoding];
NSDictionary *person = [NSJSONSerialization JSONObjectWithData:data options:0 error:nil];
NSDictionary *height = person[@"height"];
if (height) {
    NSNumber *feet = height[@"feet"];
    NSNumber *inches = height[@"inch"]; // intentional error
    if (feet && inches) {
        NSLog(@"you are %@'%@\"", feet, inches);
    }
}

This will output nothing, of course. Because of the typo height[@"inch"], the inches variable will be nil and the check will fail. Without the check, it won't crash, but it will output you are 5'(null)".

The semantics of nil in Objective-C significantly effect the design of the Cocoa. Many classes and API use nil in meaningful ways, expecting that it will allow code paths to easily and safely short out.

Now a similar thing in Swift:

let json = "{\"name\": \"sam\", \"height\": { \"feet\": 5, \"inches\": 8 } }"
let data = json.dataUsingEncoding(NSUTF8StringEncoding)!
let person = try NSJSONSerialization.JSONObjectWithData(data, options: [])

if let height = person["height"]! {
    if let feet = height["feet"]!, inches = height["inch"]! {
        print("you are \(feet)'\(inches)\"")
    }
}

This will output nothing, too, because inches = height["inch"]! will cause the if to fail. This is actually better code in this case because the language gave me a tool to leverage the new nil semantics expressively. Static typing and optionals didn't make this any safer, but it did make it clearer.

Generics are helpful when they make code more expressive and more concise. Here's a super contrived example. Notice, because Swift is good at inferring types, it almost looks like a dynamic language here.

let names = ["sam", "troy", "steven"]
let capitalized = names.map { $0.capitalizedString }

The best thing here to my eye here is that cool little word map. Having that readily available is a huge step forward. Also, Swift knows the array contains strings and so can remove some boilerplate code.

The benefit of static typing here is two fold: 1) the IDE can reason about the types and help me write the code, and 2) the compiler can optimize the output without having to rely on message passing.

Of course, this example is probably too contrived. There are ton of map implementations for Objective-C and we've been able to do the same thing easily for years. Something like this:

NSArray *names = @[@"sam", @"troy", @"steven"];
NSArray *capitalized = [names map:^id (id name) { return [name capitalizedString]; }];

It's not super important, but my quick & dirty map implementation is below. There are many great libraries that provide solid implementations, like BlocksKit. There's even a very creative implementation that builds on KVC described on a Kicking Bear post. This implementation would be anathema to a static typing fan because it's all done with strings (e.g. NSArray *capitalized = [myCollection valueForKeyPath: @"[collect].capitalizedString"];).

- (NSArray *)map:(id (^)(id obj))block
{
    NSMutableArray *array = [NSMutableArray arrayWithCapacity:self.count];

    [self enumerateObjectsUsingBlock:^(id _Nonnull obj, NSUInteger idx, BOOL * _Nonnull stop) {
        [array addObject:block(obj)];
    }];

    return [NSArray arrayWithArray:array];
}

Swift improves on this because the syntax is more concise with less ceremony and I didn't have to write a map function or take an external dependency to get it. That's a win. Being able to adopt a functional style more broadly, use delcarative immutability, and get a more optimized output from the compiler is worth the language change. If I can get a slight, potential boost in reliability I'll take that, too, but that by itself isn't enough.

Where you've got to be careful, I think is getting too enthusiastic about enforcing safety. Don't worry too much about stopping bad stuff, use these new powers to make code concise, simple, and terse.

I think you need to make an effort to keep the code and interfaces comprehensible, too. There's a clever function posted by Wil Shipley on Twitter for a clamp function that constrains anything that's Comparable to a range. It looks like this:

public extension Comparable {
    func clamp(interval: ClosedInterval<Self>) -> Self {
        return min(interval.end, max(interval.start, self))
    }
}

I have mixed feelings about this. It's probably my failing, but I found it a little tricky to interpret quickly. I had to really study it for a full minute or two to really grok why it works because of the generic types, but it's comparing an instance of some comparable type to a range of that type and constraining it to be within that range. The generic syntax makes it kind of ugly to my eye, but it goes with the territory. It lets you do elegant things like this:

let setting = readThermostatControl()
let targetTemperature = setting.clamp(68...78) // no, we won't cool to 32 degress

But, then you can also do kind of weird things like this:

"dog".clamp("elephant"..."giraffe") // um, elephant?

This contrived example is of course nonsense. Sane people don't write code this like. But sane people do write model objects that implement protocols like Comparable and sometimes it would make sense to use a clamp function on your model type. Most of the time, though, it will probably not make much sense and it might even be just plain wrong. So really, it probably just makes sense to apply it to numeric values only (int, double, float, etc.). Is it really an improvement over a macro CLAMP(setting, 68, 78)? I don't know, but it feels a little too clever. Probably mostly harmless, probably a fun coding puzzle to work out, but representative of the kind of slightly too clever stuff we sometimes see in Swift.

This is not to say that Swift is somehow unique; there's no language that's immune to writing tricky or just plain bad code. The point I want to make is that we get better with experience, exposure, effort, and some humility, not with "safe" programming languages or frameworks.

A skilled programmer will write great code in any language while an unskilled programmer will write bad code in every language. Don't try to make better programmers by forcing them to code in a cattle shoot, help them learn mastery of any language they pick up.

On Static Frameworks

There's been a lot of great and thoughtful discussion on Swift and Cocoa recently, as well as some angst and worry (some of it from me).

It really does need to be said this isn't an anti-Swift argument for me; Swift is a truly great language and we need to credit the Apple engineering team for their past and current work and trust they will continue to do excellent work in the future, as admonished by none other than Wil Shipley. His opinion carries weight.

I thought it might be useful to talk about some concrete examples of how a language and a static or a dynamic mindset can effect a framework.

Two Takes on a Web Framework

I have experience with both ASP.NET from it's humble beginnings using the PageController though various flavors of MVC, a project inspired by Ruby on Rails, and Rails itself.

It's instructive, I think, to compare the effect the mindset of the framework authors has on the design of the frameworks. Let's start with Rails.

Routing in Rails

I'm cherry picking here, but let's examine how RESTful routing is done with Rails. We start with a resource – we'll use "photos" from the Rails Guide. We would like to map a set of HTTP verbs and URLs to a controller that can handle these routes. It begins with a definition in a routes file:

resources :photos

What's happening here? There's a lot of convention at work, but we're saying that a thing named photos is being described as understanding seven different routes, one for listing or searching a set (GET /photos), one for getting a specific item (GET /photos/:id), one for creating a item (POST /photos), and so on.

The framework is then expecting there will be a class called PhotosController that has certain methods called index, show, create, and so on that take specific parameters and know how to retrieve, create, and return photos. That class will look pretty simple, too, something like this:

class PhotosController < ApplicationController
  def index
    @photos = Photo.all 
  end

  def show
  end

  def create
    # ... more app stuff
  end

  # ... more methods, but you get the idea
end

It may respond with HTML, JSON, XML or any number of other payloads. There are helper methods that can build routes and all sorts of goodies.

This is just scratching the surface, but the point is the framework is leveraging language features like symbols and mixins and dynamic dispatch to build up a framework that makes creating a web app that can do everyday things elegantly and simply.

Whether you love or hate Rails, agree with the "magical" conventions or not, I think it clearly demonstrates how someone who thinks in Ruby approaches a problem like this.

Routing in ASP.NET MVC

The ASP.NET team has been evolving the framework steadily and there have been various ways of accomplishing routing, so I'm going to just pick one that's fairly recent. My example isn't definitive any more more than my Rails example is, but I do believe it is representative and fair.

Now, the ASP.NET team could have taken the Rails approach and implemented it more or less identically. The facilities exist within the language using reflection to do exactly the same thing. After all, ASP.NET MVC was inspired by Rails. They did not, though; they approached it with a static language mindset and in the spirit of the larger .NET framework.

I'm not suggesting that's wrong. They should implement their framework in the style and accent that's natural for them. What I want to point out is how differently a static view of the world can be.

First, because this example is using attributes to decorate the classes and methods, we can let the framework discover them:

RouteTable.Routes.MapMvcAttributeRoutes();

There's nothing implied here at all, other than go inspect all the controller classes and build up a route map from their attributes. You could instead do this manually, but in that case we're not.

Now we can create a controller.

namespace PhotosApp.Controllers
{
    [RoutePrefix("Photos")]
    [Route("{action}/{id:int}", Name = "PhotosRoute")]
    public class PhotosController : Controller
    {
        [Route(Name = "GetAllPhotosView")]
        [Route("Index")]
        public async Task<ActionResult> Index()
        {
            return View(await db.Photos.ToListAsync());
        }

        public async Task<ActionResult> Details(int id)
        {
            // ... app code
        }

        [Route("Create")]
        [HttpPost]
        [ValidateAntiForgeryToken]
        public async Task<ActionResult> Create([Bind(Include = "Id,Name")] Photo photo)
        {
            // ... more app code
        }

        // ... more app code here, too

        protected override void Dispose(bool disposing)
        {
            if (disposing)
            {
                db.Dispose();
            }
            base.Dispose(disposing);
        }
    }
}

Notice we're being specific with the controller and its methods about what URL paths to respond to, why kind of verb, even how to inflate another class with data. There are some funky looking return types that are unique to this framework, but at various points in the code you'll need to think about your model types and how they are projected into and out of various representations not shown here. There are other ways of doing this, as well. This is just one way.

Is this bad? No, it's actually an improvement over previous versions that required a lot more boilerplate. But it's still kind of fussy to my eye. A lot more ornamentation, things needing to be specified and peppered around in different places, things you need to bounce to different places not shown to understand; just general brittleness and heaviness. And this is the version with some "magic" sprinkled in.

OK, So You Like Ruby More Than C# ... So What?

Yeah, that's true. I don't deny that. I also like Objective-C more than C#. Yes, I'm weird. I also like vanilla more than chocolate. My tastes aren't yours.

Personally, I love the kinds of frameworks that come from languages like Ruby. The are, to my eye, beautiful and the apps people build with them are better.

With Swift, we could design frameworks that aren't much different from the way Cocoa is now (and I'm not hung up on target/action). But, we probably won't. We'll have new minds that approach it from a Swift inspired, static viewpoint.

We could end up with things that look like ASP.NET MVC. Some folks would love that and I can't fault them. ASP.NET MVC is a perfectly rational and reasonable web framework.

Personally, I'd rather we get things more like Rails, though. I'm not sure what a beautiful, elegant, static framework would look like because, frankly, I haven't really seen one yet.

But, if anyone can do it, I bet Apple can. Fingers crossed ...

EDIT: Well, I may have found an example of what an elegant, static web framework might look like over at Vapor.

Archive - Snipposé - Game Friends - Rename Finder Items - The Fox & The Grapes