The only thing I think awards are useful for is if you understand why they are awarded. What did candidate A have that candidate B did not? The IT Pro awards appear to be a simple popularity contest. That is, random web site visitors had a set of candidates to choose from, and they chose for whatever reason. If the candidates were judged on criteria (e.g., originality, fitness for purpose, reliability, value for money) I would love to see how the winners (and losers) scored. But these awards are meaningless. There's no judging. Just popularity of clicks.What galls me is that Skype is tweeting (both @skype and @skype4biz) about how Skype for iPad was awarded "Business App of the Year" for 2011. The idea that Skype is a business app and that it is an app worthy of praise are both laughable. Unlike the folks over at IT Pro, I have criteria for these things.

What is a Business App?

To be a "business app," it has to integrate with my business. It has to do something—anything—to talk to some other IT system in my business. How does Skype do that? It doesn't. It doesn't have even the tiniest integration into anything other than its own contact list. I can't hook it up to LDAP, ActiveDirectory, or even my contacts at some other service like Hotmail, GMail, Yahoo! or AIM. If it is a standalone app with zero integration into my business, it is not a "business app." Sorry guys. You might call it "the app that is accidentally the most beneficial to business" but that's a different award.

As for it being a good app? You're joking, right? It works. That's the long and short of it. It will connect to other Skype users and you can talk to them. I have had rock solid, full-motion video calls over 3G using my iPad. I loved it. So, there's props for what they do right. Beyond that, it is an unmitigated train wreck of an application. Just look at the user interface.

Abuser Interface

Skype User Interface

I have taken the surnames off the users, but I don't have 5 contacts named Adrian or two contacts named Ajoy. There are 3 different Adrians there and 1 Ajoy, but a couple of the Adrians have more than one phone number. My Skype user interface (version 3.5.454 on an iPad2 running iOS 5.0.1) puts a single gigantic icon on the screen for each phone number. That's right, one icon per number, not one icon per person. With all this screen real estate on an iPad, they choose a layout that only puts 20 contacts on the screen at the same time! (Ironically, Apple's Contacts app only shows 12 at a time! Only 3 more than my iPhone 4) Where is the list of users? Why can't I have name down one column and phone number down the other (so I can see whether I'm dialing their work, home or mobile)? And really, in 2011, do we have the technology to understand that a single person might have more than one phone number!? This app is only a few months old. How did they write something this bad in 2011?

Contacts? We don't need no steenking contacts?

And I can't jump to groups of users by letter (e.g., can't jump to users beginning with A or M or P). I have about 300 users in my phone, which means about 700 icons on my iPad screen. Do you think I'm going to flick through these with gestures? Fat chance. So I can search at the top of the screen. That's helpful, but this UI design is still totally flawed and fundamentally useless. Can I sort by last name instead of first letter? No. Can I display "Lastname, Firstname"? No. App of the year indeed. The contacts list on my Palm Professional in 1998 was better than this.

And Skype contacts aren't shown alongside iPad contacts. They're over in their own separate page. By contrast, the way Apple gets stuff like this right is by hiding stupid details like the difference between a Skype contact and a phone contact. Think of how iMessage transparently works out whether or not it can use the data connection or must use an SMS. Skype should teach me not to care whether I'm reaching my colleague by Skype or by phone. You want to sell more Skype credit, right? Hide the distinction between real phone numbers and skype IDs a little better. Blur the lines. Let there be just one contact for "James Smith" and let it include his Skype, mobile number (for SMS via Skype credit) and other numbers. When I tap on James, ask me how I want to connect. But don't give me 2 unlabeled icons for James in this screen (one for home, one for mobile) and another unlabeled icon for him back on a different page (where his Skype account is).

Only one other feature left: history management

And the history. I can 'edit' my history, but that's a stupid feature. What does it take to delete an item from the history? Two taps no matter what. I can drag left to right to reveal the 'delete' button, and then I can tap delete. Or I can tap the 'edit' button and do two different taps: tap the red delete symbol and then tap 'delete'. But only one at a time. I can't tap a bunch of them and then delete all the ones I marked. So who cares? Why do I have two modes of deleting history items that both operate on items one at a time and both take two taps to delete an item?

This is not iDevice UI design. This is half-baked mimickry of other apps. Think of the bulk delete in the Mail app and you'll see what bulk delete is supposed to look like. Notice that the iMessage feature in iOS has a 'clear all' button and you'll see what Skype is missing in its history management. And what about deleting all history items from contact A while leaving the ones related to contact B?

And configuration options? Preferences? Tweaks to the UI to customise it to my tastes? None.

Conclusion

If I were nominating Skype for iPad as an Anything of the Year, it would be "Worst User Interface I was Willing to Put Up With Because the Features Were Compelling Enough". Or "Feature So Beneficial to Business that I Would Use It Despite It's Awful UI". This app only has a handful of UI features to begin with. And each one is amateurish and clumsy. It makes you wonder if the team members that work on Skype for iPad actually own and use iPads in their daily lives. It's hard to believe that they do.

If you look at the DHS chronology of changes to the advisory system, you can see that it came in at yellow and was mostly yellow for its first 4 years. In response to the threat posed by liquids on planes, the threat level was briefly raised to red, then it settled down at orange and DHS seems to have forgotten it. For more than 4 years it has just sat at orange—unloved, untended, unimportant.

DHS Threat Level

This is typical of security dashboards. A zillion complexities are somehow squeezed into four ordinal color values. Complex qualities like geography, industry, mode of transportation, political affiliation, population density, and dozens of other factors contribute to how likely or unlikely any given target is on any given day. And somehow, all these zillions of factors were funneled into one big color that is wrong for most people.

It's arguable that the threat level has been green, for example, for huge swaths of the American heartland. Or yellow for certain industries while orange—or even red—for others. It is not clear how anyone anywhere benefited from this dashboard during its 9-year tenure. The fact that it sat at one color for 4 years is just testimony to how impossible it is to decide which of the zillion things justify a bump up or down on so coarse a measure.

As we create security dashboards in software, or in business risk, or in functional testing, or anything else that is hard to measure but begging for metrics, let this be a cautionary tale about dialing the resolution down so far that it becomes meaningless.

My microwave oven is digital, so I enter a set of numbers to indicate the cooking time. I'm not sure if all microwaves are like mine, but mine will accept integer values that are not logical clock values. I can enter, for example, 75, and it will run for 75 seconds. If you forget, for an instant, that you're dealing with clock values, the inputs and outputs (what you type and how long it runs) can seem pretty weird.Consider the following inputs and outputs:

I decided to try to mock this up in Excel. It turns out that you can paste this into a cell, and it will give the output (number of seconds) given the input (integers typed on the keypad).

=IF(A1<100,A1,(TRUNC(A1/100)*60)+(A1-(TRUNC(A1/100)*100)))

I like this as an example of user interface design because it has some really unintuitive effects. I have some ideas how I might use this one day in training testers and security folks.

Time Warp

Facebook is savvy about time zones and the fact that its users span the globe. But notice this conversation I had with a friend. He sent it at 12:39am on October 21st. I replied 3 hours earlier. Now, if you look at our profiles, Roy and I live in the same time zone. Normally this would never happen. But, I happen to be in the UK when this happened. I think I replied a couple hours after his message, but at you'll see, it's essentially impossible to know, given what I see.

I think it has decided that I'm on the west coast. I'll have to go figure out if that's right. Without putting the time zones in there, though, it is impossible to sort out who sent what and when. You can't tell if there's an actual error in the time stamping, an error in my user profile or Roy's user profile (e.g., one of us has told facebook that we're in some weird location), or if there's an error in the localization. That is, facebook could have figured out that I'm in the UK and tried to display local times, but then goofed up the conversion to my local time zone.

I wonder what kind of software testing you would have to do, if you couldn't change the display to add in the time zone. Could this be tested in its current form? Could mistakes be distinguished from correct behavior?

I got the idea after explaining some really crappy code to my wife and how it did a ridiculous job protecting against SQL injection. I said "there must be 50 ways to inject SQL into that code." That's when she sang a couple bars and I realized it would be a great idea. Now, my singing it might not have been such a great idea, but the parody was a good idea.

I downloaded the music from a MIDI site, and then arranged it in GarageBand. Here are the lyrics:

I see your input's not validated properly
You have to check it at all tiers: 1, 2 and 3
Give me a browser and quite soon you will agree. There must be
50 ways to inject your SQL

You see it really is my business to intrude
The CTO wants to see this web app broke into
Turn on my proxy and all doubt will be removed. There must be
50 ways to inject your SQL
50 ways to inject your SQL

Try a quick hack, Jack
Add a new row, Joe
Try an insert, Kurt
Change their SQL query

Evade the regex, Rex
Encode it all in hex
Unbalance the quotes, Vinod
And change the query

Break the syntax, Max
Use a backslash, Cash
Try command shell, Mel,
And change the query

Use "one equals one," son,
Unhandled exception!
Read the stack trace, ace
and change the query

He said our application is secure against your kind
There are no simple vulnerabilities to find
I said your coders write their code like they are blind, there must be
50 ways to inject your SQL

He said our logs show unexpected funds were sent
Its probably time we started using Prepared-Statements
I said I'm glad you're seeing what I meant, there were
50 ways to inject your SQL
50 ways to inject your SQL

Break the syntax, Max
Use a backslash, Cash
Try command shell, Mel,
And change the query

Use "one equals one," son,
Unhandled exception!
Read the stack trace, ace
and change the query

Try a quick hack, Jack
Add a new row, Joe
Try an insert, Kurt
Change their SQL query

Evade the regex, Rex
Encode it all in hex
Unbalance the quotes, Vinod
And change the query

Fuzzing for Software Security Testing and Quality Assurance

Good Things

• He really puts fuzzing in context. Fuzz testing has been around for a long time, and this book gives you the full historical perspective, as well as a modern view.
• Fuzz testing is important. When Gary McGraw and company did their Building Security In Maturity Model, one of the activities that virtually everyone did was fuzz testing. Clearly we need books like this to get everyone onboard.
• Although Ari is CTO of Codenomicon, a commercial fuzz testing tool vendor, the book is not a pitch for his tool. He actually gives lots and lots of information on a broad variety of tools, including free tools. It's a complete and honest vision that is not overly promoting his company's product.
• I learned a lot of fundamentals that make a difference to how I fuzz test things. For example, I now understand mutational versus generational fuzzers. They each have benefits and you probably want some of both for good coverage.

The Not-So-Good

• I think he spends too much time talking about motherhood and apple pie security things. Things like security testing, risk analysis, code analysis, etc. There have been ample trees killed on these topics and I don't think the treatment in this book really adds to that body of knowledge. I would have been happier with just some references to the rest of the world.
• The comparisons of commercial and free tools are intermixed with all this extra security discussion. So sometimes you have to read about security metrics or some other broad topic in order to find a specific example of a specific tool.
• The authors' perspective is too much fuzzing über alles. They downplay the value of techniques like static code analysis and architecture risk analysis. Those techniques are complementary, not counter, to fuzz testing.

I like the book a lot and am glad I have it. I recommend it.

1. Certifications provide common context and vocabulary

Someone who has completed a certification, no matter how trivial, has assimilated some of the vocabulary, context, and culture that the certification tries to document. I expect someone holding a CISSP, CSSLP, GIAC, GSEC or similar certificate to speak a certain language and understand certain terms when I say them. Let's not mistakenly ascribe some loftier goal and then be frustrated when the certification's candidates don't live up to them.

2. Certifications are about minimum competence, not maximum

A certification is meant to recognize something you know. There are those who cram for a certification exam, in order to appear, for a brief moment, to know the material that the exam tests. No one thinks that the people who study momentarily are the same as those who have a long career behind their passing score. It's very difficult to design a test that cannot be crammed while staying within the bounds of cost-effective administration. Think of it this way: Mario Andretti has a drivers license in his wallet. So do I. His driving skills and mine are not comparable at all, but we both passed a test that certified a minimum competancy. He also has credentials for Formula 1 racing and years of career racing that I do not. Let's not, for a moment, consider trying to capture his experience (or Microsoft's or Wozniak's) in a test. We are just establishing minimum competence.

3. The world needs objective measures that are comparable

Ignore for a moment what value you place on the content of the exam. If the exam is carefully standardized you have a tool for comparison. If you have ever had to hire someone, you know how people make buzzword-compliant resumes today that say almost anything that could possibly help get the person a job. As a hiring official you have to sort out the BS from the actual capabilities. With a certification you have a better starting point for that weed-out process. If I see J2EE on a resume, I have a long series of questions that will get at their experience one way and another. If I see CSSLP on the resume, I know what they should know.

Now earlier I said ignore the value of the content. Now, let's evaluate the value of the content of the cert. If it has the ability to establish context and vocabulary and minimal familiarity with topics, I can work with that. If I come to discover that it has more value than just vocabulary (as a CCIE does), then I learn to ascribe more meaning to finding it on a resume.

4. Stop insulting everyone

Both the testsquad blog post and the popular anti-cert crowd make accusations of brainlessness. They claim that once you get a cert, you'll feel the rush like heroine and have to keep getting more and more certs to feed your addiction. They also claim that employers myopically focus on certs and somehow overlook the true value of the candidate. I say that the employer who overlooks a candidate's true value because he sees CSSLP on the resume would be equally duped by the long list of buzzword-compliant terminology and some good interview coaching by a placement agency. The root of that problem is the interviewer/employer, not the certification. I don't see anything inherently worse about a cert than a good coach and a bunch of buzzwords. If anything the certs are at least moderated and standardized.

5. This train is leaving the station

You can be on it or under it. The industry is attempting to create standardized comparison for various kinds of capabilities. We need to find a way to do this with integrity and value. The ivory tower people say "you can't tell if someone really knows their stuff based on a multiple choice test." There are lots of NP-Complete problems in the world that we don't think we can solve in polynomial time, yet we can apply heuristics and do various things to limit how much time we spend solving them. We need to apply the same sort of best-effort focus on quality while balancing real-world constraints. Planting our heads in the sand and saying it can't be done is not an option. The people who want the standardization of capabilities will continue to push. Those of us smart enough to know how hard this is to do can either help, or shut up. Complaining, though, is not an option.

Some colleagues of mine have recently published a book Implementing Automated Software Testing.

Elfriede and Thom were key organizers of VERIFY 2007, a software testing conference.

The vendors that give me grief right now are:

• Logitech (Harmony Remote manager)
• Red Marble Games (Democracy 2)
• Adobe (Acrobat 9)

Now what really burns me up is that I just upgraded to Acrobat 9 from Acrobat 7 on Mac OS, and Acrobat 7 WORKED on a case-sensitive filesystem. It's a regression!

To add insult to injury, Adobe's remedy for their screw-up is for me to reformat my hard drive. I see: you can't program or test correctly, so you'll just ask thousands of people each to donate many hours charitably (for free!) backing up, reformatting, and restoring so that they can have the pleasure of running your software (that they had to pay for).

If you look at their recommended solution, that is EXACTLY what they expect. I'm flabbergasted.

Finally I decided I had to cheat and see what the maximum score was. It turns out to be really easy, but I'm not going to reveal it all here and now. Suffice it to say that, with a little bit of sleuthing, you can get a word list like this:

CALYXES,SCALY,SCALE,LACES,CALYX,AXLES,SALE,EASY,CLAY,CASE,AYES,AXLE,AXES, LACE,LACY,LAYS,LEAS,SEAL,SEXY,SLAY,YEAS,ALES,ACES,CAYS,LAY,LAX,LAC,CAY,AYE,AXE, ALE,LEA,LYE,SAC,YES,YEA,SLY,SEX,SEC,SEA,SAY,SAX,ACE
And you can get it before you start the game. That means you can have the word list on your screen and all you have to do is type them in. Assuming I can type that many words in before the 120 or 60-second clock runs out, I get all the words.

What frustrates me is the fact that, even when I cheat, I can't beat my friend's score of 181. I am now able to see the "luck of the draw" factor. Some boards just aren't that valuable. They make you spend time playing them, but even if you're flawless you won't get a high score.

Knowing that took a lot out of the game for me. At least in ladder mode. Playing competitively against friends (without cheating) is still fun.