Software research shouldn’t be about the tools

It comes down to essential vs. accidental complexity, as outlined by Fred Brooks. What we research is new ways to ‘nibble’ at the accidental complexity: new languages (GO, Swift), new abstractions (Actors vs. functional programming in distributed systems), new methodologies (random test case generation). It’s what nearly every story on Hacker News is about.

But ultimately, I think most problems come down to two factors: the problem complexity itself, and the team tackling it. To me, many of the problems highlighted as software/IT failures, like the FBI registry, have nothing to do with a lack of good tools or techniques. These are ultimately management failures: scope creep, poor leadership, insufficient budget, too much budget, negative work environments, etc. It is ‘executing’ that is the problem, not the technology. How many errors have been caused by the US reliance on imperial units?

Look at this quote by a senior VP at Oracle on failures in implementing CRM projects:

[M]y comments apply to ALL CRM vendors, not just Oracle. As I perused the list, I couldn’t find any failures related to technology. They all seemed related to people or process. Now, this isn’t about finger pointing, or impugning customers. I love customers! And when they fail, WE fail.

I’d be willing to say that software engineers have all the tools they need. We need some form of continuous integration and deployment, abstraction mechanisms to simplify the problem, tests to verify our solution, version control to maintain a history of changes, and some form of requirements (whiteboard, paper, spreadsheet, what have you) to keep track of what needs to be built. I don’t even think it particularly matters how you use those tools. If you have a mature organization and process then you can all into the following matrix (James Montier via Jonathan Chevreau):

Good Outcome Bad Outcome
Good Process Deserved Success Bad Break
Bad Process Dumb Luck Poetic Justice

But just having the right tools, the good people, and a mature process is not enough to guarantee success, of course. You could be tackling a ‘wicked problem‘. You could have a team of misfits and losers. You could have a manager who refuses to accept responsibility or make decisions. Most software research does not address those issues. I’m not convinced there is any research that addresses those issues: leadership, management, sociology… nothing can help when your team lead is having a marital crisis and can’t devote any time to product development.

1 Comment

Filed under Uncategorized

Evidence in Software Engineering

This post is spurred by a line in a paper of Walker Royce, son of Winston Royce, he of the “waterfall model” (misunderstood model). He says

without quantified backup data, our software estimates, proposals and plans look like long-shot propositions with no compelling evidence that we can deliver predictably or improve the status quo.

Bold text is mine, to emphasize this notion of evidence. The question then becomes what evidence is acceptable. And here I think we get into some hoary philosophical questions concerning truth in science (epistemology, really).

I think one of the fundamental impedance mismatches in software engineering for large-scale systems, or software engineering in a systems engineering environment (e.g., airplanes, military software, ultra-large-scale software, safety-critical software) is that a number of people on those teams have a positivist view of evidence. They subscribe to the notion that sufficient “data” can show whether something will work or not. So if you design a missile system’s rocket engines, those engines either deliver the necessary thrust, navigability, etc. or they do not (actually, I think this is probably not the case in those so-called hard sciences either, but the point is that people from those domains think it is the case).

Agile software delivery works much more from the falsificationist or even outright constructionist approach. I would estimate that the majority of agile practitioners believe in the test and refine approach, where you deliver an increment, test it to determine how well the ‘theory’ of that software matches reality, and then iterate. The key difference with the positivists being, of course, that there is no a priori evidence you can use to show things will work. It is hard to do simulations in Simulink or Matlab as to how well software will perform. This is why Alistair Cockburn calls software development a cooperative game. And some people, I would say, would go even further and treat software development as a post-modernist exercise in building the reality you want to see and dispensing with evidence altogether (“if it works, it is right”). Those people probably don’t get a lot of government contracts, however.

Back to the quote: the issue remains, how easy is it to produce “compelling evidence” and what does it consist of? In Royce’s view, evidence takes the form of historical context for productivity, function points, etc. In that case we are almost measuring the team’s capability to deliver more than any particular artefact.

At some level, cynically one might say, there is a need to show the ‘evidence’ that will get the job or contract. People hire companies like Thoughtworks because they have a track record of getting the job done to people’s satisfaction. We don’t much care how long it took, or how many lines of code were written, if the software was valuable.

Which is fine, but as an engineering discipline one would like a bit more to chew on.

Leave a comment

Filed under Uncategorized

Configuring SONAR with Maven on Mac

I had this issue a few times:

  • you get SONAR installed and the web client working fine (e.g. you can go to http://localhost:9000 and see the dashboard).
  • you have a project to analyze with a Maven POM, to which you add the sonar target as described here.
  • you start the Maven run and it returns in short order saying:

Can not execute SonarQube analysis: Unable to execute Sonar: Fail to download [http://localhost:9000/api/server].

Turns out for some reason this is a problem with the default Ruby install on OSX. The workaround is to use JRuby instead of Ruby, best done with RVM, e.g. rvm use jruby. Someone mentioned this online, and I cannot find the post now, but thanks.

I use Sonar with Homebrew, by the way, which has its log files at /usr/local/Cellar/sonar/<version>/libexec/log/sonar.log.

Leave a comment

Filed under Uncategorized

The Circle, a novel

The Circle is a novel about the tech/social networking industry, where fictional company the Circle plays the role of Twitter, Facebook and Google combined. The topic is certainly ripe for the satirizing, but I didn’t think Eggers pulled it off very adroitly. Either he was going for the brutal, over the top parody like Jonathan Swift’s A Modest Proposal, or he was writing it very quickly and perhaps in anger. The characters felt a little flat and one-dimensional (Mae for example was unbelievably naive) and the conspiracy (or ‘logical extension’ perhaps) was hard to believe—Americans are fiercely proud of being independent and private, so the idea that they would willingly join in with mandatory Circle membership felt off. 

Some points I thought were well made. For example, the skewering of the “campus culture” of tech companies which privileges the already-privileged while the drivers of the busses and local inhabitants are worse off. The technocrats who think technology presents a simple solution to all problems (witness the Gates Foundation and educational testing, for example) are an unfortunate reality today. Parts of the book raised good and thought-provoking questions about understanding social media tradeoffs. Some of the reasons why privacy is important are really poorly outlined (but perhaps the question is wrong on its face—why should I justify my need for privacy?). There really are things that technology has improved dramatically—every time I use Skype with my parents I realize this. Tracking my friends on Facebook is a nice thing to do and brings them more into my life as well. 

I appreciated that aspect of the novel, and it has given me a new perspective on being a ‘user’/’product’ of social media tools. Overall, this is a topic that cries out for a sustained and subtle approach and I felt The Circle was too facile and, for the Valley cognoscenti who most need it, too easy to dismiss. By the way, there is a certain irony in posting my thoughts about this novel online, but doing so on my own blog feels somewhat different. I hope. 

Leave a comment

Filed under Uncategorized

13 Great Software Architecture Papers

In the paper “The Past, Present and Future of Software Architecture”, the authors (Philippe Kruchten, Henk Obbink, and Judith Stafford) have a sidebar in which they list their selection of “Great Papers of Software Architecture”. I’ve tried to collect these papers and links thereto for future reading. Here is a bibtex file for full citations. I’ve also included the parenthetical comments of Kruchten et al. in italics, and my comments in bold. Unless otherwise noted I’ve linked directly to the PDFs (please let me know if a link breaks).

Foundations

Precursors

Architectural views

  • D. Soni, R. Nord, and C. Hofmeister, “Software Architecture in Industrial Applications,”This article introduced Siemens’ five-view model, which the authors detailed in their 1999 book Applied Software Architecture
  • P. Kruchten, “The 4+1 View Model of Architecture,”Part of the Rational Approach—now known as the Rational Unified Process—this set of views was used by many Rational consultants on large industrial projects. Its roots are in the work done at Alcatel and Philips in the late 1980s.

Process and pragmatics

  • B.W. Lampson, “Hints for Computer System Design,” This article and the next gave one of us (Kruchten), a budding software architect in the 1980s, great inspiration. They haven’t aged and are still relevant.
  • J.A. Mills, “A Pragmatic View of the System Architect,” paywall
  • W.E. Royce and W. Royce, “Software Architecture: Integrating Process and Technology,” —This article articulates the connection between architecture and process very well—in particular, the need for an iterative process in which early iterations build and validate an architecture. Not online, sadly.

Two more for the road

Leave a comment

Filed under Uncategorized

Virtual Conferences

The virtual conference has many benefits in terms of GHG emission reductions, cost savings, and accessibility. Jon Pipitone and Jorge Aranda co-organized a very small event with me in 2011 around climate change and software. We used Skype and instant messaging to connect climate researchers with software researchers, but my feeling afterwards was that virtual worlds like OpenSim were the way to go. 

It’s cool to see Crista Lopes has managed a relatively large-scale event. 

Leave a comment

2013 September 11 · 13:31

Knowledge and complexity

Somewhat inspired by +Rob England, I tried a mapping of Rumsfeldian terminology to Cynefin (yes, i know this predates the SecDef!).

Known knowns – either a simple or complicated case. If simple, we do it routinely. E.g. landing an aircraft in good weather.

Known unknowns – we have a plan for accommodating it. E.g. landing an aircraft in Edmonton with high crosswinds.

Unknown unknowns – we are in a complex domain and we have to see how things should work with experiments e.g. edge of envelope flying. One of the things that the Kennedys made clearer for me is how experiments have to be well conceived, in particular, by controlling variables properly (see their SysEng Journal paper).

Unknown knowns – we didn’t realize we could plan for this but now that we sense it, we can delegate to existing routines, or perhaps adapt to it. That pathway is available but unused. The example would be .. looking through the manual and realizing the system can actually do this (maybe the Apollo 18 case? There they did the exaptation that Snowden often mentions.)

Snowden would no doubt criticize my limited understanding, but there is some use in seeing how these frameworks co-exist, for me at least.

Update: Dave Snowden replies with a pointer to his HBR article with Cynthia Kurtz in which “knowns” are discussed. Summary: simple contexts are the domain of known knowns, complicated contexts are the domain of known unknowns, since experts are required. Unknown unknowns are the domain of complex contexts. Interestingly, they categorize the Apollo 13 case as being in the complex context. In the sense that there was no clear answer, that makes sense, but to me, it also highlighted that idea of unknown knowns: that is, these skilled engineers did “know” the answer (since the astronauts survive), but not consciously. So we could perhaps characterize that as relying on the “expert within”.

Leave a comment

Filed under Uncategorized