Wake up folks - architecture is YOUR responsibility

If you want to make SOA work in a heterogeneous environment, why would you want to limit yourself to one technology, one approach? You’re buying right into their strategy of locking you into a platform. That’s only good if you sell platforms. Wake up folks - architecture is YOUR responsibility, not that of some vendors hawking middleware!

Ron Schmelzer of ZapThink

Simple

I descended into a simple blog link vortex - a couple notables:

Bill de hÓra: All successful large systems were successful small systems

Dare Obasanjo: My Website is Bigger Than Your Enterprise

Ryan Tomayko

Sam Ruby: Radical Simplification

Proposed Standard

Oh just a proposed standard huh?

So what does "Proposed Standard" mean? That might not sound very stable - "Proposed" - but that term does have special meaning in the IETF. The best way to explain it is that there are plenty of other protocols that you use every day that are "Proposed Standards", for example: WebDAV, TLS, LDAP, SMTP, SIP, and IMAP. Of course, if you want to know more about the IETF's standards process, they have it documented, in RFC 2026

Joe Gregorio talks (now at Google) about GData and how says: This is great news for us because the Google Data APIs are built on the Atom Publishing Protocol. The current Google Data APIs are based on early versions of the AtomPub specification and now that AtomPub is a Proposed Standard work will begin on getting all of our Google Data APIs compliant with RFC 5023.

Push & Pull Naming Standard & the Uniform Interface

The uniform interface is great. Resource name + GET, POST, PUT, DELETE & you are done. This is an amazingly simple / powerful constraint of the REST architectural style.

I spent a little time today thinking through a naming standard for our integration architecture. We need to support Push & Pull. This is essentially REST/AtomPub & Messaging.

For REST, I liked what RESTful Web Services had to say:

. . . So which is it? URI as UI, or URI opacity? For once in this book I'm going to give you the cop-out answer: it depends. It depends on which is worse,for your clients: a URI that has no visible relationship to the resource it names, or a URI that breaks when its resource state changes. I almost always come down on the side of URI as UI, but that's just my opinion.

It references URL as UI (Jakob Nielsen) and Universal Resource Identifiers -- Axioms of Web Architecture (Tim Berners-Lee)

URI Templates seem like a natural fit for specifying URI standards & URIs that are in use.

What about on the Push side (i.e., traditional MOM/XMPP) - what is the analogue? Is there one?

I toyed with the idea of using the same verbs (GET & POST at least), but I think that the semantics are different enough with messaging that it is either too clever or just lame to use a subset of the same verbs. The main issue is that GET isn't idempotent in messaging. I guess you could use GET for Browse, DELETE to pop off a queue, and POST to send a message, but it just seems too contrived to me. These are different styles - why break people's head with foolish consistency?

In the past I have had good luck with NOUN.VERB in messaging which is similar to REST in a way if you standardize on it.

Other ideas are NOUN.INPUT, NOUN.OUTPUT; NOUN.REQUEST, NOUN.RESPONSE.

Any other ideas / experiences?

Sweet Slides: The Web: Distributed Objects Realized!

OOPSLA 2007 The Web: Distributed Objects Realized! (Stu Charlton and Mark Baker)

Good paper(s) too:

REST is clearly not the "end of history" with regards to network-based software architecture. At smaller scales, with somewhat different desirable properties, other styles, such as Remote Data Access, or Event Based Integration, continue to solve important organizational and technological challenges. Beyond the current iteration of the Web, future requirements will require a re-evaluation of REST's constraints and desirable properties for a new set of requirements. This may lead to relaxing some of REST's constraints, or an introduction of new constraints (Extending REST for Decentralized Systems). Regardless of the direction this may take, a continued means to successful future systems architecture will be the discipline of objectively evaluating constraints, and the properties they induce, for the new generation of global scale, networked-based software systems.

Via Stefan Tilkov

Update fixed broken link to presentation. Also added this image as it caught my eye:

I distinctly remember the "Object Web" phase of distributed computing. I remember being a participant in it. I bought into CORBA. I remember saying things in 1998 like, "we need more - we can't be constrained to HTTP GET and POST in the browser. We need to be able to bust a socket and talk to an ORB for a richer experience" I was kinda clueless then. At least I had a lot of company - everybody including Lotus Domino which I was integrating with when I made that comment was busily adding CORBA capabilities to there whos-its and whats-its. Well everybody besides MSFT and friends, but that is a different story.

Almost 10 years later, we still just have GET and POST in the browser, but XHTML 5 will eventually fix that. It turns out you can get pretty far with just GET and POST. You can get even farther by just discarding distributed object / RPC technology all together and using the REST uniform interface.

QCon - Connecting SOA and the Web: How much REST do we need?

Like Patrick, I'm very excited to head to SF for QCon.

It will be great to meet Stefan Tilkov, Steve Vinoski, Pete Lacey, and Dan Diephouse in person. We'll no doubt have a lot to discuss.

History of HTTP

While at Powell's Technical this morning, I picked up HTTP The Definitive Guide.

It has a couple of good links on the history of HTTP:

Why a new protocol? Tim Berners-Lee (1991)

A Little History of the World Wide Web

Web Architecture from 50,000 feet

What the software of the future looks like

A nice post from Jacob Kaplan-Moss:

Let me try to explain what I’m talking about. You want an API without having to write a line of code? It’s called curl, and it ships with your MacBook. And just look at how simple the APIs are in your favorite language.

The wins go deeper than APIs, though. Think about what it would take to add load-balancing to CouchDB. I’ll give you a hint: perlbal. Or what about adding a transparent caching layer? Try Squid.

HTTP is the lingua franca of our age; if you speak HTTP, it opens up all sorts of doors. There’s something almost subversive about CouchDB; it’s completely language-, platform-, and OS-agnostic.

Look, CouchDB may succeeded, and it may fail; who knows. I’m sure of one thing, though — this is what the software of the future looks like.

---

Patrick and I were talking about this a bit yesterday. It is tough for some in the middleware world to get this. It will come with time. It is a bit nuts that your integration architecture really is just the judicious use of HTTP, well designed URIs, and data formats. How that architecture is realized (CouchDB, Squid, perlbal, etc. etc.) really won't matter over time.

If you can't answer yes, yes, and yes to: completely language-, platform-, and OS-agnostic when it comes to your integration architecture, you are doing something very wrong.

Via Simon Willison

Ubuntu 7.10 Install Today + Garden Shop + Semantic Web

I am going to do a couple 2-3 things today:

1. Install Ubuntu 7.10 (Gutsy Gibbon) Check out the screen shots
2. Go to the garden shop and get some ferns
3. Buy & start to read one of these if they are not at Quimby Warehouse (code for not in stock at Powell's Technical down the street)

XMPP Slides

Check out the nice slide deck on Peter Saint-Andre's blog.

Bloatware vs. Pairing Knife

An interesting post from Eric Newcomer on the HPTS (High Performance Transaction System) Workshop:

For example, one of the Web guys commented that they did not want to have to buy the whole set (i.e. complete product) when all they really needed was a paring knife, yet the only choice they are offered is to buy the whole set.

. . .

"Bloatware" is definitely part of the issue. Ironically so are many of the things a lot of us have worked to define, create, and promote during the past two or three decades around guarantees of atomicity and isolation. None of the Web companies use distributed two-phase commit, for example. They only use it to deque an element and update a database in the same local transaction. So much for all that work on WS-TX! ;-)

. . .

It will be very interesting to observe over the next few years the extent to which the ideas and techniques in the custom built solutions become more widely adopted and incorporated into commercial products. One of the inevitable questions, as raised during the discussions, is how broad the market is for such things as Google's file system and big table, or Amazon's S3 and Dynamo.

---

The next few years will indeed be interesting as we enter the post WS-* era. It will be interesting to see which middleware companies survive. For one I wonder if Google & Amazon will more completely enter the enterprise integration business by productizing some of their core technology.

PDX Software

The Silicon Forest blog has a brief interview with Jim Zemlin (The Linux Foundation - formally OSDL):

You have to look at the open source movement as a global phenomenon. My day literally starts with a call from Moscow and ends with a call from Beijing. From that perspective, geography is less important in this incredible global collaboration over the Internet.

Having said that, I'd like to look specifically at Oregon and where I believe Oregon provides a leading role...starting with Linus...I think it's important to Oregon that Linus lives there. I think that is a real indication of the importance of Portland as a place where real leaders in the open source community reside and do their work.

. . .

There is opportunity for places like Portland because of this fact, that it is a global phenomenon, to really attract top people to the area, either to be close to their peer group, or just based on the fact that it's a more affordable and superior standard of living.

---

It will be interesting to see what Portland looks like in 20 years. Come to think of it, it will be interesting to see what lots of places look like in 20 years. I am naturally most interested in Portland since I will in all likelihood still be here involved in the creation of software.

I see more small - mid sized software companies in Portland's future. Also small satellite development centers for mid-large companies as the world continues to flatten. I don't see many major companies moving here or anything like that. Portland has a decent amount of talent in open source & software more broadly.

The biggest thing Portland really has going for it is people (especially creative people) will typically move here in a heart beat as it is such a great / unique place to live.

I don't see Portland ever being a Tier 1 city or major tech hub. But clearly over time (20 years or so) SF, Portland, and Seattle will continue to morph together. I see a very bright future for software in Portland.

Distributed Scrum

InfoQ has a nice presentation:Planning and Maintaining the Rhythm of Distributed Scrum.

Summary At Agile2007 we heard the tale of a distributed Scrum project with 50 people in 4 continents. BMC Identity Management decided to build their next generation product, including architectural changes and component integration, using Scrum to handle the uncertainty of their product's requirements.

Will the technical debt consume us?

Pete Lacey quotes Anne Thomas Manes:

The problem is caused by the root culture of IT — project-driven funding models, a cobbler’s kids perspective on investing in infrastructure that helps IT (rather than a particular project), and a propensity to never decommission applications. IT systems have grown organically for the last 40 years. They’re a mess. It requires a fundamental change in the way IT operates as a service provider within the organization.

Here is the post on the Yahoo service-oriented-architecture list.

She is of course correct.

And all the while the technical debt is piling up. Maintenance is accounting for more and more of the IT budget. Soon there will be nothing left for new development.

There are of course solutions to this problem, but they require a lot of discipline and vision.

I think the discipline and vision are much less costly & more effective than the annual madness of budgeting in your typical large company.

Update 12-OCT-2007

I don't know why I didn't think of this last night, but Patrick quoted 12 Questions with Mary Poppendieck just yesterday in our internal wiki:

The first step in moving from forecast-driven projects to feedback-driven... is to change the measurements. The book "Rebirth of American Industry" by William Waddell and Norman Bodek makes a good case that the measurements imposed by traditional cost-accounting methods are the biggest impediment to the successful implementation of lean manufacturing. Similarly, I believe that the measurements imposed by traditional project management methods are the biggest impediment to the successful implementation of lean development. In particular, instead of measuring variation from plan, we need to start measuring the delivery of realized business value.

AtomPub RFC - RFC 5023

James Snell notes that AtomPub is now RFC 5023.

I think that AtomPub has heaps of potential.

Ubuntu 7.10

It's coming soon.. 11 days.

Ubuntu rocks. The new version is going to be interesting to watch. Ubuntu is already fantastic. But it seems like this could be a break out release with all they eye/feature candy.

ESB Spaz Out Excerpts

A couple 2-3 excerpts from the ESB spaz out that caught my eye. I have largely sat this one out since playing a small part in starting it. Perhaps I'll say something soon, but for now, here are a few quotes that caught my eye. I'm too spent from a long week to say much more.

Steve Vinoski: Reactions to the ESB Question

Eric Newcomer comment: I tend to think of the IT world as divided between systems designed before the Web, and those designed to include it. The mindsets are very different, as you point out, but I would add that the pre-Web mindset is kind of driven by mainframe centric designs - I like to think that the issues with existing middleware (and perhaps binary languages) is due to the fact we always felt we had to design in all the features/functions of mainframe based systems in order to entice enterprises to move those apps to standards based systems.

There is a lot of truth to this. I live with a lot of big iron. Interestingly, AtomPub & the whole pull based feed approach to integration is somewhat similar to batch processing. Perhaps it can help bridge that gap a little bit? Hey, that would be cool - Atom feeds from VSAM files & AtomPub from CICS. Maybe, but it is the batch cycle itself that has a strangle hold. Batch cycles that have been nurtured for 30+ years are very difficult to untangle. Things are really bound up there.

The Enterprise Cool URI will save us yet.

Steve Vinoski: The Degenerating ESB Discussion

Dan Hatfield:Honestly, I see the ESB as primarily a political thing. It allows for a greater degree of control on delivered solutions. In large companies, we don’t often do architecture - we do politecture…The politics drive the architecture. Not the way it should be…but that’s the way it is.

Politecture! Ouch. Sad but true. I walk this line on a regular basis. I push the envelope as far as it can go, but politics are ever present.

More Vinoski: Another non-technical way to look at it is from the viewpoint of Clayton Christensen’s classic book, The Innovator’s Dilemma. For quite a few years now, we’ve seen a series of sustaining innovations in the “object/service RPC” line of descent originally popularized by CORBA and COM, both of which built on earlier RPC, distributed object, and TP monitor technologies. RMI, EJB, SOAP, WS-*, and ESB are all offspring in that line, and there are surely more to come. I feel that REST, on the other hand, fits the definition of a disruptive innovation perfectly (and if you’re too lazy to read the book, then please at least follow the link, otherwise you won’t understand this at all). The proponents of the sustaining technologies look at REST and say, “well it can’t solve this and it can’t solve that” and voice numerous other complaints about it, precisely as Christensen predicts they would. But Chistensen also explains why, at the end of the day, any real or perceived technical shortcomings simply don’t matter (and in this case, they’re mostly perceived, not real). HTTP-based REST approaches have a lower barrier to entry and are less complex than anything the sustaining technologies have to offer, and REST is disrupting them, whether all the smart folks pushing ESBs like it or not. It’s not a technical issue, and there’s no amount of technology the non-REST tribe can throw at it to stop it because it’s based on how markets work, not on the technical specifics.

Internet Machine

Tim Bray: The Intimate Internet

But when the next big thing comes along (and I love this business, because I know it will) you won’t have to rely on the professional noticers to tell you because it’ll touch your life directly.

I love this business too.

This internet-machine is wicked cool.

Pretty Good ESB Spaz Out Underway

The "always-insightful" Patrick Logan has triggered a nice sized blog spaz out on ESBs it seems.

Good stuff is being discussed. Cool. The ESB Question - Steve Vinoski of former IONA fame.

More insight from Patrick: Properly Striking a Balance Between Shared Agreement and Decentralized Execution

There is plenty more, but you can find the rest pretty easily.

Camel Cast

James Strachan has a Camel Cast.

Ed (co-worker) and I messed with it a bit a couple weeks ago. It is approachable for newbies in that it is just code. I haven't really gotten that excited about the whole DSL thing, but an Enterprise Integration Patterns DSL sounds nice.

When Ed and I messed with it, it seemed young, but headed in a good direction. There is a lot to be said for simple - just download a jar file or 2 vs. adopting a whole platform. I know for a fact that there are a lot of people out there that would like to stop writing this type of code, but either can't or don't want to adopt a whole platform to do it.

It loks like the Camel docs might be getting rev'd a bit. I particularly liked seeing links to unit tests in the docs.

Good docs rock. Matt Asay says it results in more sales for open products.

These abstractions make it easier, see.

I came across this on time spent in "modern" Java web dev. Funny stuff.

I rat lol'd at the Spring bit and the Hibernate bit.

Via Buko Obele

Cool Enterprise URIs & Mule

Patrick Logan expounds on my Enterprise Cool URIs post.

Ironically, Ross Mason commented on Patrick's post & Patrick responded with what we've been discussing lately at work. The ironic part is I spent the better part of yesterday messing with Restlet and Mule (Ross's product).

My point of view on this topic has changed significantly over the years like Patrick's. I have been pro-CORBA, pro-EAI, tolerant of WS-* (couple months where I built a couple of them in 2001), pro-message broker, pro-ESB, pro-JavaSpaces, and most recently pro-please don't hurt the web

I think that if you spend enough time with middle-ware, you eventually conclude that the best approach is to not hurt the web as much as you can and embrace a more-or-less-peer-to-peer model (i.e., how the web works). A year and a half ago, I preferred the message-centric to the service-centric approach. I have concluded that at the time I was misguided in how I thought I was dealing with the fallacies of distributed computing. I don't think there is anything wrong with messaging & there is a place for it for sure - I just think that the dominant integration protocol should be HTTP. I used to see messaging in the center, now I see it on the edges. HTTP should be in the center.

It is probably important to note that I am talking about coarse grained integration ... the integration of large domains (e.g., sales, manufacturing). I've heard this called the "federated ESB" approach although I wouldn't call it that. I've also heard it called the "ABC" model (application, business unit/domain, corporate). I'm focused right now on the communication at the corporate level - across domains rather than within them. I think that HTTP makes sense everywhere, but for lower level integration (A & B) there may be other choices (I still think that conceptually, you can't beat the Tuple Space concept for A - large applications).

Mule is an ESB, but that term is essentially meaningless today. MuleSource also refers to it as an "integration platform" and an "Enterprise Service Network". Mule allows and encourages any-2-any protocol "mediation". This is good. Mule can certainly be used in a way that does not hurt the web.

I have more learning on Mule to do. I was a little overwhelmed in working with it yesterday (lots of jar files, lots of XML config, examples that didn't help with what I wanted to do, docs that didn't answer my questions - typical newbie stuff). I did succeed in getting it configured to do what I wanted it to do after going down a number of rat holes & now have something to build on this week.

Mule may be a good runtime choice, but I agree with Patrick that an ESB isn't your architecture. Your architecture is things like reference data, meta data, the services & events you expose, your URI naming standard, security, and what is on the wire. You want to build these things so they can stand the test of time.

The appealing thing to me about hiding as much of this as possible behind a good URI naming scheme that doesn't change & HTTP is that I know for a fact that the URI & HTTP will be thriving in 10 years. I can't say that about any integration vendor let alone any integration vendor's current product set.

Taming the Traps of Traditional Thinking

Nice article: Mind of the Innovator: Taming the Traps of Traditional Thinking. It talks about the "seven sins of solutions" & how to tame them.

This is tragic & very true. We all see this:

IDEA loops. IDEA is an acronym for Investigate, Design, Execute, Adjust. it’s a codification of the human learning cycle...the one that starts disappearing around age 5, once we enter the formal school system. that’s when it becomes about the right answer and not the right question.

I have been fortunate in that I have been affected by this less than most. Some of it is personality type, most of it is probably due to really good parents who encouraged me to try things and not be afraid to fail.

We need to stop thinking about innovation as an outcome, and start thinking about innovation as a process. we need to move from innovations to innovation. Because as a practical matter, innovation, problem-solving and learning employ the same iterative process—blending supposition, logic, creativity and reflection

Experiments show that creative revelations come when the mind is engaged in an activity unrelated to the issue being addressed, and that pressure is not conducive to creative thought. recent research demonstrates that the ultimate break—sleep—actually changes our mind’s perspective.

There are pros and cons to being a light sleeper (me). I sleep 6 hours a night unless I'm exhausted. I personally need to be more disciplined in turning things off. Most of the time, software related "work" doesn't seem like work to me because I genuinely enjoy it.

There is this too: Enter the irrational fear of failure. Backing off is counterintuitive. it somehow feels wrong, like preemptive surrender. it’s scary to ease up, because we may lose our steam, or we may abandon hope. we get anxious when the answers aren’t so forthcoming, and we begin to doubt our creativity, abilities and intelligence, fearing that if we take our eye off the problem even for a moment, we may lose the energy we’ve invested.

I definitely have the fear of losing steam - I'm generally not afraid of failure though. I'm a "damn the torpedoes" type. This too has pros and cons.

decisions: "follow the decisions you made" vs. "they were made"

I saw a link to this on Dan Creswell's del.icio.us:

Train-Wreck Management by Mary Poppendieck.

The article is referenced on InfoQ. The Poppendiecks are my favorite methodologists. I have yet to be disappointed with Lean thinking.

I highlighted some sections that stood out to me:

Exhorting people to "be careful," "try harder," and "work smarter" is not useful if individuals have little effect on results. Rewarding or punishing people for outcomes that are not under their control can only result in discouragement - or in gaming the system. Instead, chronic problems must be fixed by finding their underlying causes and addressing these effectively. As Deming points out, this usually involves changing the system - the way things are done. And according to Deming, it is management's job to change the system.

Deming: All of the empowered, motivated, teamed-up, self-directed, incentivized, accountable, reengineered, and reinvented people you can muster cannot compensate for a dysfunctional system.... A well-run organization with well-functioning systems allows people from top to bottom do work of which they can be proud.

"There is something called standard work, but standards should be changed constantly. Instead, if you think of the standard as the best you can do, it's all over. The standard work is only a baseline for doing further kaizen. It is kai-aku [change for the worse] if things get worse than now, and it is kaizen [change for the better] if things get better than now. Standards are set arbitrarily by humans, so how can they not change?

"When creating Standard Work, it will be difficult to establish a standard if you are trying to achieve 'the best way.' This is a big mistake. Document exactly what you are doing now. If you make it better than it is now, it is kaizen. If not, and you establish the best possible way, the motivation for kaizen will be gone.

"We need to use the words 'you made' as in 'follow the decisions you made.' When we say 'they were made' people feel like it was forced upon them.

Standards are not about how work should be done, but how work is being done. You don't want the standard to be too perfect, because that leaves no incentive for workers to improve their standards.

That is, workers - led by their team leader - do many rapid experiments, find a better way, agree on the improvement, quickly document the new way, and use it. When a standard is improved, the decision for the change must be made by the people doing the work, so they won't feel it is being forced upon them.

When Deming said "change the system", he was talking about changing the complex, interrelated processes used to get work done. Deming believed that changing the system is management's primary job, and in order to do this, managers need competency in four areas: 1. Appreciation for the overall system in which work is done 2. An understanding of variation - and the true relationship between cause and effect 3. Constant pursuit of learning (improvement) through designed experiments 4. An understanding of the psychology of people When all of these areas are balanced and working together, great things can happen.

Ready. Fire. Aim

David Christiansen sent me a good article by Tom Peters on systems thinking and innovation. I couldn't agree more with it. It is astounding how much time is wasted with excessive planning.

Tom's post reminds me of the book The Myths of Innovation. I read it earlier this summer.

I'm a big believer in failing fast and learning. If you want to innovate, you plan a little bit, create a whole team, give them a wiki, foster a collaborative/transparent culture, set a short deadline, try something, measure it, and do it again until you are done.

I whole heartedly agree with this excerpt: I am an unabashed fan of MIT Media Lab guru Michael Schrage—particularly his book Serious Play. His principal axiom: "You can't be a serious innovator unless you are ready and able to play. 'Serious play' is not an oxymoron; it is the essence of innovation." And, in turn, the heart of his serious play is ... fast prototyping: "Effective prototyping may be the most valuable core competence an innovative organization can hope to have." His intriguing connection, which makes all the sense in the world to me, is that true innovation comes not from the idea per se, though it guides the work, but from the "reaction to the prototype." In fact, in a surprising number of cases (the majority?) the collective responses to a host of fast prototypes reshape the original idea beyond recognition—or lead one down an entirely new path.

A more succinct way to sum up how innovation works is Ready. Fire. Aim. Tom's article says he heard a Cadbury exec call his approach to product development. This phrase is often used as what not to do when it is exactly what you should do.

Update 01-OCT-2007 Tom has some follow up posts that are well worth reading: Systems Thinking II: My Summer Vacation and Systems Thinking III.

Here is a great quote:

"For C-sakes, quit drawing those f-ing maps and run some experiments, quick and dirty, and see if anything you are babbling on about actually works or makes the slightest bit of sense in the real world as we know it. And after you've done your real work, then you are welcome to write your 'complete theory of everything.'" (That was close to the actual script, minus many more f%^*s and about 25 minutes of elaboration.)

Enterprise Cool URIs

Cool URIs don't change.

We are working on some fairly large scale integration at work.

As you might expect, we are trying to not hurt the web.

As most older "enterprises", we have many many aging systems. But the "domains" or large groupings of our systems are well known & shouldn't change too much over the next 10-20 years.

I'd rather bet on the URI than any ESB vendor.

I started working on a naming scheme for our cool URIs today. I think that if we get that right, we'll be heroes.

With a URI naming scheme that doesn't change you get a very different, very simple view of your systems. Sure, there may be madness today behind those URIs, but over time that madness will hopefully start to go away. But your URIs will stay the same. A simple vaneer on madness that slowly becomes sane. Yay Cool URIs that don't change.

Pair Atom Restlet

I rewarded myself after a long week of teeing-up the next phase of a big project with some pair-programming with Patrick.

We did some messing around with Restlet & Atom.

It was a pleasant afternoon.

Restlet is a great little API.

Atom is a great little format.

Linux and open source software pay off for PayPal

Linux and open source software pay off for PayPal: When Scott Thompson left Visa to take the CTO role at PayPal in 2005, the Web company's data centre surprised him. "Wait a minute," he recalls saying, "they run a payment system on Linux?"

"I was pretty familiar with payment systems and global trading systems, but I just scratched my head when I came here," Thompson says. With his history of working on IBM mainframes and large Sun Solaris systems, the PayPal approach to computing seemed alien, especially for a company whose core mission was dealing with money.

PayPal runs thousands of Linux-based, single-rack-unit servers, which host the company's Web-presentation layer, middleware and user interface. Thompson says he quickly saw the economic, operational and development advantages of open source and Linux technology. He now sees no other way to do it.

"When you're buying lots of big iron, as I did in other places I've worked, your upgrade path is $2 million, $3 million at a clip. You just had to buy big chunks of stuff to scale," he says. "Here at PayPal, our upgrade path is 10 $1,000 no-name servers, slapped into the mid-tier of the platform. And we just keep scaling it that way. It's unbelievably cost-effective."

Via Matt Asay.

GOSCON 2007 in Portland

I got a note from Deb Bryant regarding GOSCON. I went last year and it was great. I'm shocked at how much has happened in a year.

• The week after I heard Larry Augustin speak at GOSCON, Compiere was forked. Less than a year later, a VP from Oracle took over Compiere hoping to turn it around.
• JanRain was just getting started - I saw Jason McKerr speak at GOSCON. He had recently left OSL which was pretty high flying at the time. I remember him saying, "I'm speaking at an OSS conference, but our products aren't OSS. We did a lot with OSS at OSL." I don't remember much more about what he said. A year later, their CEO has moved on. I have no idea how they are doing besides that. But when that happened, it reminded me just how hard it is to start a company & how hard you have to work to make good software products that the market needs & will pay more $$ than it costs to make. Looking at the JanRain website, I don't know where the $$ will come from. I do think that OpenID is as good an idea as any though for identity. I get really tired of registering and remembering my password. I just don't know how they will monetize it given some of the stuff going on in that community. I hope they do though - clearly some talent there. And I have a natural bias towards all Portland tech companies.
• Bradley Wheeler gave the closing keynote at GOSCON - it was incredible.
• Stuart Cohen gave a speech as the CEO of OSDL (now The Linux Foundation). He now runs The Collaborative Software Initiave (CSI) - it is conceptually similar to Brad Wheeler's success with universities.
• Open Source was a key focus of my daily work (along the lines of Brad Wheeler & CSI)- now it is just one fairly small aspect of it. Open Source has come a long way in a year yet has a ways to go. IMHO it is an unstoppable force. But lest we forget, it only makes up 2% of the software market. My personal opinions around open source have changed a ton in the last year. I remain convinced that it is the #1 disruptive force in the software market, but am a little more realistic about where it is and where it is leading.

Anyway, GOSCON is on again this year (year 3) GOSCON 2007, Oct 15-16 in Portland. Deb says: This year's keynotes include: Andrea DiMaio, Vice President and Distinguished Analyst, Gartner Research; Jim Zemlin, Executive Director, Linux Foundation; Skip McGaughey, Director of Eclipse Ecosystem, Eclipse Foundation.

Roy Fielding Video

Stefan Tilkov links to Roy Fielding presentation video. It is about 30 minutes long & well worth being late for work over ;)

I wish I could go to ApacheCon to see the latest in person.

Message Queue Panic

Now Dare Obasanjo is joining the spaz-out re: messaging, push vs. pull and such.

Also, Stefan Tilkov is momentarily a binary messaging bigot.

XMPP is IETF'd. It has been around for years (started in 1998 according to Wikipedia).

AMQP is a young upstart. Perhaps it will break out. I'm just saying it will be hard. It is really hard to get a protocol going. Time will tell I suppose.

At the end of the day, IBM MQ Series/WebSphere MQ still has over 90% of the messaging market last time I heard so in terms of application integration within the "enterprise", I'm putting my money on that continuing for the foreseeable future.

Perhaps overtime, AtomPub, XMPP, and HTTP can put a dent in that.

The whole pub/sub business may be an opening. MQ Series isn't known for that. They have a new 100% Java version that I presume does a better job. XEP 0060, AtomPub, etc. could be alternatives.

In the short term, I think that ActiveMQ has the best chance of taking on IBM MQ Series/WebSphere MQ in terms of the more classical messaging stuff. But then again, I guess that may be ok with IBM as they support Geronimo which includes ActiveMQ.

Seagull Architect

Speaking of David Christiansen, he recommended a good book recently that I have been reading: Manage It! Your Guide to Modern, Pragmatic Project Management.

He dropped "Seagull Architect" in some meeting we were in a few weeks ago. He got that from the book.

A few excerpts from Johanna Rothman (author):

I've worked on several projects where the architect was like a seagull. He swooped in, dumped a lot of poop in the form of PowerPoint pictures of the architecture, and left as soon as possible. He didn't stick around for the hard part of the project: making the product work in the architecture ore evolving the architecture so that the product could work by the time of release.

. . . But if your architect is overly fond of drawing programs and not fond of writing code and can't really answer the developers' questions about how to make the parts fit into a coherent structure, you don't have a real architect. Eliminate that person from your project . . .

Yep, PowerPoint doesn't compile.

I have been accidentally guilty of this before. My reason was being spread thin, not that I can't sling code. I hated doing that to the team at the time. And it was true, the team really suffered. Ultimately I removed myself from the project and they were successful.

The $100MM Canonical Model

A co-worker of mine in Indianapolis, IN David Christiansen has a good post with The $100,000,000 Canonical Model.

Lest we forget, there are no sliver bullets.

There are two extremes in my mind when it comes to data formats and integration architecture:

1. Big Ball of Mud
2. A canonical model / data exchange architecture will solve everything

The pragmatic approach, IMHO, is to start small and build these assets over time via projects. The wrong approach is to go off for months/years to create it. Clearly, an asset like this needs to be as simple as possible, embrace change, be accessible, and based on Atom. (ok the last bit was bias).

GData AtomPub Podcast

I just listened to The world of Google data APIs. It is 42 minutes long. I took notes.

Here is the agenda:

• What "Google data API" actually means (the parts and pieces)
• What Atom, Atom Publishing Protocol, and other tech behind GData are all about
• What GData adds to the mix on top of Atom and APP
• How Atom compares to RSS
• What are ETags? And how can they help me?
• Why REST, the style, was chosen for these APIs
• Where REST makes sense, and where it doesn't. Resource driven vs. RPC.
• What the first GData APIs were
• How the killer app of syncing data with Google Calendar
• How you actually use the APIs? What do they need to learn? What tools do we give them?
• Can you write APIs that implement the same GData APIs?

Notes: Atom is an IETF standard in case you didn't know. RSS isn't.

Google has been working with Atom for over 2 years.

AtomPub provides a basic REST API - Google thought this was a great starting point. Atom leaves query to the student. Google uses URLs to do this. You can have output sorted etc. Atom doesn't say anything about this.

ETags were unfortunately not implemented. They chose a version number in the URL so that multiple people can write to the same entry. Plan to implement ETags in the future.

Are ETags magic? No, a lot simpler than they sound. A little string that tells you the version of the entry. Just a great way of making caching work & making the web more efficient.

Chose Atom because of momentum. And sold on REST. Not SOAP because the web is based on REST. Much easier for devs to learn REST than SOAP. SOAP require tooling. REST is simply manipulating tables of entries.

What are disadvantages of REST? AtomPub is just an implementation of the REST style. Difficult to map certain types of operations to REST - translation API example: send text & send back a list of options & then feedback how to improve the translations. Document centric request / response that was never saved on the server. Just a straight RPC call. REST is about manipulating resources.

How about transactions? Each request is essentially transactional. REST not concerned with multi-resource transactions.

How do devs use? Can use curl if they want to - some do create apps with shell scripts and curl. Have a Java, .NET, PHP, Python, Objective C API. Contributed APIs: Lisp (Patrick!), Ruby, Flash in development. It's just XML over HTTP - it's NOT THAT HARD!!

Authentication - "Client Login" is pretty straight forward auth. URL for uname/passwd where you get a token back that is your identity. Authentication for Web Apps - "Auth Sub" used for "on behalf of" type stuff. You can grant access to other web sites. You control who has access to.

Can I use Google Auth? They are meant to be open standards. Google legal hasn't signed off yet, but licensing will be worked out soon.

What is this "Kinds" business? Entries get passed around a lot - kinds concept is Atom categories to tag each entry with a "kind". Just gives more semantic information to computers (clients).

AtomPub Google Interop Event? 12 devs from different orgs came. Great success. Google's basic AtomPub worked fine. Google custom auth scheme was more problematic. After that everything worked smoothly. A lot of the impls built around AtomPub introspection document - Google doesn't use them much. Will in the future.

Any GData / AtomPub tips? Google has a particular approach to designing APIs. AtomPub good at certain APIs. Things that map well - RPC not one of them. On occassion Atom Entry is just a pointer to the real data (e.g., photos). Prefer to put data in entry as much as possible. There is a lot of art/style/parsimony to it. You can achieve a lot with 1 feed with a lot of query parms. Have to review APIs & ensure that they have good clean concepts: this feed clearly includes these types of elements.

WADL? Haven't found the need for it.

Atom can be tough to get at first, but once you do it is amazingly simple & then applicable to many. Very good programmers who don't understand AtomPub gargen/language can. Concepts are very simple. Feeds, entries, links to other entries. Very simple mental model once you get it. All APIs make the same. Very powerful.

Google working with the IETF on improving Atom/AtomPub. Introduced batch model that increases efficiency a lot. Auth. Teams in Google are very ambitious - hope to make publically available as drafts that can become standards.

Some talk of JAY-SAHN (JSON).

Viking Bus

Erik Onnen (will answer to "Viking") has a good post on Apache HTTPD & using it as your integration bus.

Day by day I am making a cleaner break with my MOM bigot history.

There is clearly a place for messaging, but I think it's future is on the web. XMPP is of course perfectly positioned for this.

Fixed

Matt Asay writes about a case against a SI (actually my first employer - it was called Andersen Consulting then):

[T]he U.S. Department of Justice is suing Accenture for allegedly receiving kickback-like payments from technology suppliers it recommended and/or implemented at DOJ. The alleged fraud was a collusion with big-name IT suppliers (e.g., HP, Sun) and smaller vendors (e.g., Vignette) to defraud the Government.

IT land is a very interesting place. It never ceases to amaze me. There is very interesting behavior in this market. There are all sorts of vendors all chasing the same Fortune 500 - 2000. There is tons of money, yet there is none. Everything is easy, yet everything is impossible. To survive/thrive in IT you need to develop many different types of skills. You must, however, maintain your integrity above all else.

Note: Obviously I have no idea if there is any merit to this case. All of the people I worked with and Andersen Consulting had heaps of integrity. I just felt the desire to beat the integrity drum today and saw this and leapt on it.

Viking Async Sighting

The Viking (aka Erik Onnen) spied AsyncWeb.

From AsyncWeb home page (bold emphasis mine):

The majority of today's popular java HTTP engines are built around the J2EE Servlet Specification - and are inherently synchronous in their operation. The typical order of play is that an individual HTTP connection is mapped to a thread of execution which reads data from the underlying socket, parses the request and forwards it up through the web container to the target Servlet - blocking until a response is completed.

As processing latency (as described above) increases, a real scalability problem is encountered: To increase throughput, it is necessary to increase the number of connection threads. This, however, is not the best way to scale - and depending on the JVM / operating system employed, having a very large number of threads can become a big problem very quickly.

A real problem then needed to be solved: How can we obtain a very high throughput - whilst dealing with high processing latency - without requiring a potentially unbounded number of threads? An initial area of investigation was to provide a NIO transport connector for an existing HTTP engine. However, existing popular HTTP engines built around the J2EE Servlet Specification have a blocking synchronous architecture throughout. Swapping in a NIO transport to such an engine does not solve the problem: We'd still need to scale processing threads in order to push requests up through the container.

What was needed was an http engine which provided non-blocking behaviour throughout - and so work began on writing such an engine.

Looks like AsyncWeb has been heading for Apache MINA for some time. Apache MINA is a network application framework which helps users develop high performance and high scalability network applications easily. It provides an abstract · event-driven · asynchronous API over various transports such as TCP/IP and UDP/IP via Java NIO.

The things I drawr don't come true!?

Wait, those gigantic diagrams on my wall aren't going to compile with the next version of my bitchn' uber IDE? But the vendors said it was just around the corner!!

"Words and diagrams are all we have to work with. Mr. Klein. We need to live within their limitations. Seek not a single diagram to capture your enterprise architecture. Seek not a single sound bite. It is a hyper-dimensional thing you are grappling with. All you can do is capture fleeting glimpses of it with words and 2 dimensional views. Do not start with the wiring diagram. Mr. Klein. End with that as the least important view of your enterprise architecture. One view amongst many and a not-very-important view at that." Sean McGrath

Via Stefan Tilkov.

Dan Diephouse & Abdera (AtomPub)

Yay cool beans Dan.

Spring & Abdera

Both please

Dan Creswell picks up on the INATT business.

The problem is myopic views that focus on either only technology or only business. My point is that the lines are blurring. And the right approach is a balanced view.

If you spent millions and a year on a "build it and they will come" approach to deploying an ESB for your SOA platform, drink 3 shots of "It's Not Not About the Technology KOOL-AID" ASAP.

If you are ignoring the fundamental changes occurring in technology & the relationship society has with technology, drink 3 shots of "It's Not Not About the Technology ALONE KOOL-AID" ASAP.

CouchDB

I can't remember who's blog pointed me at this, but it is very interesting: CouchDB: Thinking beyond the RDBMS.

There clearly is a need for something like CounchDB. All the impedance of mapping document data to relational is quite tiresome.

I don't think that we'll ever get rid of relational databases all together, but at the very least it would be nice to have something for the cases where it makes no sense to map to a relational database and you want to effectively retain your documents. This could be great just to effectively store service interactions. Is it really just JSON or can I stick XML in there too? JSON is great and all, but there are an awful lot of XML documents out there.

I thought document storage like this was a great idea going back to 2000-2001. I was working at eXcelon (now Sonic Software) at the time. They had a product named XIS that was an XML database built on top of eXcelon Objectstore. It's issue in general at the time was scalability - I am not quite sure what happened to it after Sonic Software (Progress) bought eXcelon.

REST Conversation

Benjamin Carlyle has a good post on simplifying the introduction to REST here.

I love the question/answer style blog post. I have to do that sometime.

INATT

INATT - It's Not Not About the Technology

INATT is pessimistic and self-defeating, even if it's not intended to be. It denies that there can be improvements, incremental or radical, in the ability of technologies to accomplish important goals. I disagree categorically with this. . . .

I think I may have even said this last week. In that case it really wasn't about the technology. But more often than not, It's not about the technology alone. as Andrew McAfee says.

IMHO technology is going to steam roll a lot of people and companies in the coming decade. Many are in denial. We are in an age where generations and generations of platforms have accumulated and we are drowning in technical debt. And we have generations of people that are accustomed to doing everything on computers, cell phones, and the Internet. These people will be the majority in 10 years.

Of course business execution aside from technology is crucial. The problem is that technology is becoming more and more crucial to business execution. INATT denies the freight train coming towards many businesses.

I predict that more and more companies will be coming to terms with the fact that it is about the technology like Amazon did. What happens when there is the Amazon of your industry?

Via Stefan Tilkov.

Atom & Pub/Sub Revisited

Tim Bray picked up on my Pub/Sub vs. Atom & AtomPub? post.

During the last year, I have not been doing that much messaging, but the previous 7 years I did a substantial amount of it - both point-2-point and lots of pub/sub. I absolutely don't think that Atom / AtomPub by itself will replace messaging. I don't know enough about XMPP to know if it will replace traditional messaging some day. I am a big fan of AMQP, but don't know if it is viable yet or will be viable soon. Tim is dead on that PSB is an impressive corpus of work

My point, was only that Atom & AtomPub appears to be a very good format / protocol to do business events with a polling model. Having done a lot of pub/sub, I have done a lot of clustering. The buffer that Tim describes is not so simple when you get lots of clients and lots of consumers. That is where flow control starts to appear (if you really want to guarantee message delivery) and where the dark corners of fail over and clustering rear their heads. It is nothing against messaging per say - just how it is. My point really was that even "guaranteed" messaging is often not truly guaranteed. And that for the right usage scenario the poll model is simpler and more appropriate.

Furthermore - push models can be appealing to developers because it feels cool (has felt very cool to me) - even simple. My point is just that this isn't typically really the case.

It really just comes down to your requirements. For example message rate, how quickly your event sinks need to process, etc. For where I am thinking right now, high level business events don't typically need to be delivered more than every 10 minutes - perhaps ever minute. I'd much rather deploy some simple HA web infrastructure & Atom/AtomPub than AMQP, TIBCO, SonicMQ, ActiveMQ, etc. to meet those requirements.

In my experience with pub/sub style integration architectures when you are pushing information amongst major domains (e.g., party/customer information), there is always the nagging question if you missed an event - or how do I deal with drift. This is because even though it is supposedly guaranteed messaging, there is always a risk. I've seen people try to capture events and persist them so that they can be replayed in the event of this type of a problem. Or more commonly, check the master system every month to ensure that the systems are synchronized. An Atom feed seems like a good way to avoid this failure all together - in that rather than pushing a copy of the event to each interested sink, the sink just reads the feed. Sure the client has to be a little bit more intelligent, but it just seems more deterministic to me. But I have more to learn.

Atom Service Directory

I re-read some of the Atom specifications on a plane today.

I haven't been as excited about a data format since creating a mostly proprietary, but somewhat based on ACORD one at a Canadian insurance company in 1999-2000.

I still have a lot to learn about ATOM, but it seems to address all of the pain points I have seen in integration data formats so I'm optimistic.

I obviously can't be the first one to think of it, but wouldn't Atom / AtomPub make a nice simple Service Directory (think UDDI, but with legs)? Just a feed with entries for each service instance? Categrories and other meta data? You could dynamically register services via AtomPub - perhaps services would have to re-register every so often. Is there something like this already?

I also looked back at some of the blogosphere spaz-outs regarding Atom yesterday. I have more to learn, but perhaps it is best that Atom get's a head of steam before embrace and extend sets in anyway.

Here's to hoping that Atom stays simple - seems to be a beautiful format.

XML Schema Languages

Ok, so Atom for the document format, what about the XML Schema Language?

According to Tim Bray (and others), it is no contest - RELAX NG wins.

The last two canonical models I was involved in used XSD. I have a hate hate relationship, with XSD, but I'm not yet convinced that RELAX NG is the path forward for me right now. I need to get my learn on. The biggest issue, really, is that there is a lot of XSD out there - a lot of schemas that I may want to fork into my canonical format are already in XSD.

In my experience, the basic 25% of XSD is ok - it is the other 75% that has a lot of very dark corners. That and the never ending evil of code generation from schema. Here are some other benefits of RNG over XSD according to Wikipedia.

I also have to figure out where Schematron fits (or doesn't fit).

What tools are people using with RELAX NG and Schematron that they are happy with? I see Topologi. Last time I was involved in this we used XML Spy. It is a pitty that it does not seem to support RNG?

What about runtime support - what works well for Java, .NET, Ruby, and z/OS (the last one should be fun).

Any experience in the trenches?

Canonical Model Format & Atom

It has been awhile since I thought a lot about canonical models. I have been involved in a few over the years.

I'm thinking about them tonight.

In creating one today, is there a reason why you wouldn't use Atom instead of say SOAP or more likely your own proprietary solution? I re-read Pete Lacey's classic The S stands for Simple to remind myself why I left SOAP behind.

SOAP isn't even on my radar these days, but perhaps someone can convince me otherwise on why it belongs in the heart of your integration architecture? Of course you have to interoperate with it, but you can certainly keep it at the edges (i.e., integrate with legacy web services, packaged apps that only expose a SOAP interface that you want to use).

We are talking about your lingua franca here - there are few things in my opinion, more important to an integration architecture than a canonical model - you want to get this right.

SOAP started simple enough - it just became an industry and now it is fairly hard to separate the wheat from the chaff. Here's to hoping that Atom stays simple.

I kinda doubt that many people are using SOAP as a part of their canonical model. I would have to bet that they are using something they made up. From what I can tell, Atom seems to give you what you need out of the box - most people will need to add some more standard elements to it, but it has all of the basics so why reinvent the wheel?

Jive Funding

Wow, Jive took some VC funding.

A couple of these guys are my friends - they work their butts off. Great guys, great company.

Congratulations Jive!

Amnesia & Terminal Gravity

These brewers are not easy to find. But they are incredible.

Terminal Gravity (Enterprise, OR).

Amnesia Brewing (Portland, OR)

After a long day yesterday, I had one of each at Henry's Tavern last night :)

Weak Sauce: RESTful Flex Angst

I think this report from Erik Onnen on Flex and REST is pretty weak sauce.

I'm surprised Flex doesn't have a real HTTP client built into it. You can't do REST with just GET and POST. Being dependent on the browser is pretty lame - is this driven by some sort of other requirement or some security sand box what-nots?

I guess that is what we get for hurting the web.

How are others dealing with this?

Pub/Sub vs. Atom & AtomPub?

I came across an interesting article by Sean McGrath entitled I'll push and you pull. The mashup approach to application integration.

I posted about Push vs. Pull in the spring. I'm a sucker for Lean thinking so the title of this article got my attention.

Sean says:

Now here is the kicker. The web - with all its concomitant bits'n'bobs from XML to RSS/Atom to AJAX - is an extremely good platform for pull-centric design. On the Web, if you try to pull some piece of information and something goes wrong, well you just pull again and again until you get it or give up. Nothing fancy. Just brutish repetition. Something machines are extremely good at. If you want to look at information from yesterday, you just go to the URL that contains yesterday's information. Nothing fancy. Just a simple naming convention that includes dates in URLs.

I have done a fair amount with pub/sub over the years (complete with clustering etc., fail over strategies, etc.). I like it, but it is not without it's challenges. If only there was real guaranteed delivery etc. If only there wasn't "flow control". If only fail over always worked. If only messages didn't get trapped, etc. All I'm saying is having implemented several large implementations of pub/sub (EDA whatever you like), I know that it isn't easy - that nothing is really "guaranteed".

Clearly Atom can't replace ALL pub/sub use cases, but for every day integration architecture where you want business events / EDA why can't we use Atom feeds? In an extreme case, you might have an event sink requesting the feed every 10 seconds - in most cases every 10 minutes would likely be fine?

Who is doing this today? Any lessons from the trenches?

Update I meant to read the article earlier linked off of this post by James Snell. Just did. Dang. It is a sample of using Apache Abdera.

Update 20-AUG Bill de hÓra has a pretty $$ response - thanks Bill.

Update 21-AUG Dan Creswell has some thoughtful comments. When I blog I often do the x vs. y thing - but clearly the truth is in the middle. I used to be a push bigot and have just learn the hard way how difficult it is to achieve. Clearly, not everything can be pull - of course it is a mix. In case anyone is curios, I'm thinking of inter-domain integration (e.g., getting customer additions/updates/deletions to many interested systems) with this line of thinking rather than some sort of intra-domain integration (e.g., trading system where there is massive high performance pub/sub). You have to choose the right tool for the job. More and more for me, simplicity is winning out. This is just an evolution of my thinking - a year ago I was still a MOM bigot.

Bell's Oberon

Speaking of beer, my family had a fair amount of Bell's beer in Glen Arbor.

Oberon is delightful.

Bell's is far and away my favorite non-west-coast-microbrew.

Deschutes Brewery Down the Street

My favorite beer of all time is Mirror Pond.

It is certainly the beer I have consumed the most since I got out of college. Sadly, the beer I probably have consumed the most of is The Beast.

I am pretty happy that there will be a Deschutes Brewery down the street from me this coming Spring.

Note: the picture above is their new label - finally they are getting rid of the crappy current one. It isn't that great either. Oh well you don't drink the label very often. I still think the first 2 generations were best: (source)

Second Generation

Original

New blog - Erik Onnen

Hey cool, one of the smartest people I know, Erik Onnen, now has a blog.

Yipee, more of the world can start to learn from once of my favorite collaborators.

Glen Arbor

I just got back from a Herrick family vacation in Glen Arbor, MI.

It was a lot of fun. I really needed a break - I have been very busy.

I used to go there every summer as a kid with my family. It was nice to get back there - I hadn't been there in about 20 years.

I can be a bit of a bigot about Oregon and how it is superior to just about anywhere. But it is tough to beat the Great Lakes in the summer. My mother will love to hear that (no mom this does not mean I am moving back to Michigan any time soon - sorry :) ).

Here is a picture of Pyramid Point that I took.

Don't Move to Portland

It Rains All the Time - you will hate it.

I'm rUl busy, blog is not getting a lot of action. Hopefully it will pick up soon.

Met Eben Moglen

I had my month made today.

After work I was lucky enough to meet one of my heroes, Eben Moglen.

I talked to him for about 10 minutes. It was a thrilling experience.

I am very predictable when I meet famous people. I typically say something odd. There was no exception today with Eben. I said, "You are one of my heroes!" not once, but 3 times. And then I shouted "Keep up the good work." when I left.

Oh well, I don't care - I hope he does keep up the good work.

Enough Web 2.0

After attending the OSCON O'Reilly Radar: The Executive Briefing today, I have had enough Web 2.0.

I attended this part of OSCON last year as well.

Although it was well worth the time today, I believe that last year was much better. I think I was a lot more "bright eyed" last year - even after attending OSCON for the previous 3 years at the time. I was starting out in a new exciting role and building a new team. I was very thirsty for this type of thing then. This year, the team is in place, it is great & we have already been through a lot together. And we are busy trying to deliver, so maybe it is just a mindset thing.

One thing is for sure - I have had enough of the Web 2.0 crap. I never liked the phrase and I am thoroughly sick of it. I'm with Eben Moglen - 10 years from now, Ajax and Web 2.0 will be "thermal noise".

Last year I was blown away by Tim O'Reilly. This year I wasn't. Don't get me wrong, he is a really bright guy, but this year he just didn't seem to have the same zip (come to think of it, none of the O'Reilly people did). I will say this, Tim was very gracious in talking to Eben Moglen - I could totally see most people flipping out in that situation. Eben admitted that he was intentionally making a spectacle of himself and Tim to draw attention to important issues - you could see that making the receiver (Tim) upset.

Here are some highlights and lowlights from my opinion:

Highlights

• Eben Moglen vs. Tim O'Reilly - I'd have to vote for Eben - but who really could debate Eben and win? Tim was gracious though. Update (hour or 2 later): After reading a few other reactions and thinking about it further, I don't quite understand why Eben attacked Tim that way. I'd like to hear an explanation. Was it warranted? Sure doesn't seem like it?
• Firefox Extension Ecosystem
• Doug Cutting, creator of Lucene, Nutch, and hadoop
• Marten Mickos, CEO of MySQL - a real breath of fresh air after Oracle's Mike Olson - what a solid thoughtful dude.
• Ohloh.net - as an enterprise customer of course I like this
• OpenCV - man I feel dumb about the code I sling after watching this. Wow.
• Karl Fogel - on Congress and version control and Subversion & centralized SCM vs. Decentralized

Lowlights

• Dave Morin from Facebook - a big "so what!?" from me
• Roger Magoulas Director of O'Reilly Research - last year he had way more zip. This year his slides were broke and he kinda tanked. May just be that the novelty of it all wore off from last year. Last year, I was blown away by his presentation.
• Matt Asay vs. Mike Olson from Oracle - Sorry Matt! I think that I was just more annoyed with Mike's POV, although he did recommend a book that sounded interesting: Non Zero - The Logic of Human Destiny, so he wasn't that bad. Anyways, wasn't awful, I just have heard that stuff before. I wish Matt spazzed out more like I know he can.
• Jim Zemlin (The Linux Foundation). Sounds like a nice enough guy, but I liked Stuart Cohen better - but I know him so that isn't fair. I don't really get what they do now, other than pay Linus and other core committers which is good enough for me.
• Mark Shuttleworth - Ubuntu - as a big Ubuntu fan I was a bit underwhelmed. It wasn't bad, but it just wasn't covering areas I was terribly interested in. Mark is a good dude though. He even has Ubuntu in his will - it will be well taken care of should his next space trip not go well. Sweet! Maybe I'm just jealous that I'm not him ;)
• Andy Oram, O'Reilly - Open Source Documentation - I was lost and bored and the slides didn't work.
• OpenID - I was just tired at this point. It did cause me to check in on JanRain. That was the first I heard that their CEO changed. Reading Scott Kveton's post on resigning reminded me just how hard start up life can be. I have no doubt that I will do that again at some point in my life, but man does it pack a wallup. Good luck JanRain and Scott.

Compiere Change

I saw this on one of Matt Asay's blogs.

It is nice to see Compiere getting back on track.

I noticed last October that they got forked.

I also read this essay by their new CEO (from Oracle).

I found it interesting. It will be interesting to see if an old dog can turn it around. He seems like he has done his homework.

OSCON - Eben vs. Tim O'Reilly

Wow, Eben Moglen and Tim O'Reilly really got into it today at OSCON.

They are two people I respect - debate == good.

Couple quotes:

Eben: I was reading you last week - talking about we need to revisit what freedom means. (this is what has Eben really hot)

Eben: Gave 10 year licensing opportunity to have conversation in public policy concern and not licensing concern." "Go back to rights and responsibilities.

Eben: We gave you 10 years - I just bought you enough time so that we can fix.

Eben: We are going to have some social charters and ecological rules of the road

Basically, Eben is pissed because he believes Tim is a self-promoter (Web 2.0) and that he is making money on the back of OSS and is now talking about how we need to focus more on freedom.

Interesting conversation to say the least.

OSCON

I am very excited to attend the O'Reilly Radar: The Executive Briefing. Last year it was fabulous. Definitely the highlight of the conference for me.

I'll post my thoughts from the briefing.

I'm so buried with work this year that I decided to skip the rest of the conference. I could have used a really good geek out as I am rather fried, but maybe next year. The rest of the team is there so I'll just catch up with them.

Rocky Votolato

My friend and co-worker Ed went to see Rocky Votolato last night at Doug Fir in Portland. It is a nice small venue.

If you haven't heard of Rocky you should give him a listen. He is dang good. He is even better live.

Laws of Software Development

Sarge sent me the Laws of Software Development.

Good stuff.

Adobe AIR

I went to the Adobe AIR bus tour last night. I love events like that! Wow, that was fun.

AIR (the proper name for Apollo) looks quite promising

It does break your head a bit to see JavaScript accessing the file system, however.

AIR looks simple to get started with - and it will run on Winders, OS X, and Linux. That is great - there really aren't great options today in doing this. You need a special class of develooper that knows a lot of stuff to do this well today.

AIR is cool because it uses technology that developers are already familiar with - this will make it approachable.

AIR could really be big if Adobe executes well on it.

Looking for a Development Manager

We are looking for a world class development manager in Portland, OR.

If you are interested, please see the job req..

Here are a couple of the traits we are looking for:

• Technical - experience with current technology (e.g., Web, Java, Ruby, Linux, REST, integration styles)
• Leader
• Motivator
• Hard charging - make stuff happen
• Track record of delivering
• Have desire to be part of growing something
• Visionary
• Collaborative - able to work with others to make the vision real
• Experience with judicious application of Agile and or Lean techniques
• Heavy integration experience
• Strong communication, political, diplomacy skills
• Financial Services experience (desired, but not required)

If you have questions or are interested, please contact me.

Document Management File Formats

I have been thinking a bit about file formats related to document management lately (aka enterprise content management).

Specifically around forms (think about the printed policy you get from your insurance company).

You may just get this in the mail and laugh that it is still often in ALL CAPS.

But this stuff is enormously complex.

I'm trying to figure out the core document formats. I figure if you make good choices there, you might get it right with the details. If possible I'd like to use open standards, but I'll settle for published friendly standards - there are many 100% proprietary options out there that frighten me (who wants to be beholden to one vendor forever with no way out short of complete re-implementation?).

There are two types of file formats that have my interest right now:

1. form file format
2. archival storage format

There are lots of options here of course. And lots of pros/cons. I have a long way to go until I have an opinion. The right answer may be more than one.

With form file format a couple options are perhaps:

SVG, XForms, XHTML, XFA.

With archival storage format options could be:

PDF/A, ODF, OOXML (gasp), more TIFF and AFP.

As I learn more / develop opinions I'll try to share. If you already have some, please do to the same :)

Apache CXF 2.0

Dan Diephouse announces Apache CXF 2.0.

Dan is an impressive dude.

But then again, Dan is from Grand Rapids, MI. People from there in general are pretty impressive.

Now I have been known to bash WS-*. But CXF (XFire) does more than that. And Dan is a pretty thoughtful dude who just stays above the fray and slings solid code.

Anyhoo, congratulations Dan.

Everybody buy Dan a beer at OSCON.

Updated to "CXF" from "CFX" 04-JUL

Distributed SCM

I have been reading about Mercurial on blogs for over a year.

I saw this on Dan Creswell's delicious.

Good stuff - I have lived similar pain - who hasn't. Anyhoo, I want to try it - I want to do a million other things, but I just can't work any harder w/o snapping! I wish there were more hours in the day, I must find more ways to get more done.

Anyhoo, this link sucked me into Rockstarprogrammer blog. Nice blog and what not.

Hey look, it isn't just me, Hibernate is hard. Rockstar dude (Dustin) explains it exactly. But it is too late, his project is likely over committed to it and he'll just have to fight to the death on the next project to avoid it.

As if you didn't know, I think OR tools are awful. I think they open Pandoras Box on anything but simple projects. You want a cache? Get a cache! You want object to relational? Suck it up and write SQL. Don't mix these concerns.

Hitting the Big-D Tomorrow

I'm going to hit the Big D tomorrow.

Yay.

I managed to completely fry myself out this past week.

We made some pretty key progress so it may have been worth it, but as of 4 PM yesterday I have been trying to just kick it.

Yay just kicking it.

I downloaded some music for the ride to the river including:

Brad Paisley, Rocky Votolato (incredible), Plain White T's (iTunes made me), Tift Merritt, The Magnetic Fileds, The Frames.

Hopefuly I will hook into some beautiful redsides.

Out at the plate

Hey that was cool.

I watched the Oregon State baseball game on TV last night.

They repeated.

Best play was when they threw the guy out at the plate - that is the coolest play to watch in baseball. Really cool. It goes like this:

1. Hitting team hits a screamer to the outfield and base-runner on first or second tries to score
2. Fielding team outfielder runs and gets ball
3. Outfielder chucks ball to relay guy
4. Relay guy chucks ball to home plate
5. Hitter either tries to get around, out run, or bumb rush catcher

I don't watch much sports anymore, but that was cool.

Yay Oregon State baseball team.

Open Source Software - Sales Guy Bell Weather?

You can say a lot about enterprise software sales guys (I happen to like them), but one thing is for sure - they know where the money is at.

This is by no means a statistically significant sample, but I know a lot of these guys (mostly from college) and many of them either just got out or are trying to get out of this business. At least the traditional commercial proprietary business.

They all want to get into either SaaS companies or commercial open source companies.

I think this is pretty significant.

I am on the software buying side these days. I know how hard the commercial proprietary business was, is, and how much harder it is going to become in the coming years.

If the guys who know where the money are at understand this and are shifting to companies outside of commercial proprietary what does that say?

2 Year Old Gets Out of Crib

My 2 year old has overcome the shackles of her crib.

She wanted to get out so she climbed out.

She then proceeded to reenact how she did it for me and her mommy.

Quite a milestone.

Here is to her overcoming future obstacles in life.

OR - ActiveRecord

Via Bill de hOra I saw this: this on ActiveRecord at Twitter.

Couple 2-3 quotes:

One major downside of having an object-relational mapper is that you don't always control what goes on behind the scenes.

This pretty quickly brought us to an obscure corner of the ActiveRecord source (three cheers for source code!), where it became apparent that Rails was doing these gigantic loads from the database every time we saved even a single field in a related object. There are a bunch of mitigating circumstances that mean that this bug doesn't get triggered all the time, but it's still really really bad.

You mean insane SQL isn't just a problem with Java based OR tools?

I think Ruby is great. I think that ActiveRecord is great and do think that it has it's place.

My problem is that every time I have been associated with an OR tool (Toplink, EJB CMP, Hibernate) the exact panic described above has ensued. In my experience OR tools have been like very bad debt. They are shiny objects that make initial development a bit simpler & then make you pay pay pay down the road.

Now I don't have much information on Twitter, but I'd think that it would be the type of system that you would want to write SQL by hand for. You can either get some syntactic simplicity from OR tools and defer how the SQL gets generated (and wait for panic to ensue) to an OR tool or you can write SQL (a really simple language) and be in control.

Control isn't always required - and I am a big fan of Ruby on Rails - just sayn' choose wisely - there is more to life than concise model to database interaction.

As for the Three cheers for source code line, +1MM. Never ever use an OR tool that is not OSS. That is just asking for it.

I am going to get a breakfast burrito now.

Good Hearted Man

My wife thinks I am a good hearted man most of the time.

I just caught Tift Merritt on Austin City Limits after finishing a particularly long week.

She rocks/countrys out pretty good.

And her song "Good Hearted Man" is pretty solid.

Who is going to OSCON?

So hey - OSCON is coming up. Happily for me it is 2 blocks from our office here in Portland.

I just registered.

So did Patrick and the other people we work with (2 other guys and girl).

I would love to rip a beer and talk about stuff with anybody who wants to.

Blogging & Wiki Syntax

I really hate that when I type a post I have to type out the HTML. I spend hours a day inside of a wiki using wiki syntax that makes typing things like links, bulleted lists, and tables trivial. All of this takes way too much effort in creating a blog post. I use blogger - blogger you need a tab for wiki some wiki syntax - preferably the wiki syntax that I use at work. There must be something out there like this? Some client or some such?

Anybody know?