Panic From Fuzzy

Moved

2010-09-11T20:47:00.000-07:00

I'm officially retiring Panic From Fuzzy. It's moved to blog.mikeherrick.com. See you there!

Hiring Web Core Developers

2009-12-12T07:19:00.000-08:00

Collaborative Software Initiative Inc. is hiring Core Developers to contribute to the development of some exciting products. We are looking for great people who love software as much as we do, who take pride in their work, enjoy being part of a distributed team, and thrive on the CSI Collaborative Approach:

Transparent development model inspired by Open Source
Community Source: Developers are directed in their efforts by Subject Matter Experts
Collective Vision: Breakthrough Thinking
Team Building
Lean and Agile Software Development
Pragmatic Product Management
STAR Award Best Practices for Product Support
Defined Governance Model

See The Collaborative Approach for more information.

We use a wide range of web technologies in our products including JRuby, Ruby, Java, PHP, Postgres, MySQL, NoSQL based databases etc. The product development teams we are currently hiring for have a mixture of Ruby and PHP and have exciting feature areas. Think you are one of us? Contact me at mike ATSYMBOL csinitiative.com

Looking for another TriSano™ Core Developer

2009-03-28T09:32:00.001-07:00

We are looking for another Core Developer to join our CSI TriSano™ team. CSI TriSano™ is an open source, citizen-focused surveillance and outbreak management system for infectious disease, environmental hazards, and bioterrorism attacks.

You can live anywhere in North America.

Our team is distributed and very tight. We have members of the team in Portland, Ore, Boston, MA, Cleveland, OH, and Indianapolis, IN., and Washington DC. We use open source and Lean techniques in our pursuit of greatness.

Check out CSI TriSano™ Enterprise Edition Core Developer Job Description.

To learn more about the project see: TriSano™ Community Site

To learn more about the product see: TriSano™ Product Page

Email me if interested: mike@csinitiative.com

TriSano Perf Notes - JRuby & Postgres

2008-12-04T20:15:00.000-08:00

We've been doing a little bit of rudimentary perf testing for TriSano lately on our test infrastructure at OSL SA'd by Lance Albertson. We haven't turned a whole lot of knobs yet, but we are seeing some rather exciting trends in Postgres and JRuby.

In upgrading from Postres 8.2 to 8.3 we saw some crazy improvement with inserts. Check it. Selena Deckelmann our favorite Postgres person (and trusted consultant) explained that this was due to HOT. Nice! When we first ran it, we didn't believe it so we re-ran it a couple times before we believed it. Much rejoicing on that one.

With JRuby, were seeing impressive improvement between 1.1.3 and 1.1.5 in terms of speed & memory usage. I love watching big honkn' GCs. Check it 1.1.6RC1 isn't showing a whole lot of difference for TriSano in our testing yet. JRuby 1.1.6 Final is due next week. The JRuby project is on a pretty wicked trajectory. Exciting stuff. We really can't say enough about the JRuby guys.

TriSano uses Rails - we haven't had the chance to upgrade to Rails 2.2 yet, but we'll do that next year. That will be another exciting perf test to watch in terms of memory footprint given the multi-threadedness of 2.2. Exciting stuff.

1 Sales Engineer, Please

2008-10-16T20:23:00.000-07:00

Collaborative Software Initiative is on the hunt for a killer experienced Sales Engineer.

Interested?

Email us.

Update 18-OCT:

This position is actually called "Technical Director" its a key role in both project forming and productization. The intersection of business, market and technical strategy ...

TriSano @ GOSCON

2008-10-10T08:04:00.000-07:00

Come hear all about TriSano at GOSCON.

I'll be speaking with David Jackson, from the State of Utah.

Here is our session overview: Applying Open Source Methodologies to Build Public Health Software Collaboratively.

Some exciting stuff is happening in government facilitated by collaboration and open source. Come hear about it.

TriSano Project Annoucement

2008-08-21T13:08:00.000-07:00

TriSano is now open source.

Check it out!

TriSano.org

This is an exciting day for the TriSano team & Collaborative Software Initiative.

From the project annoucement:

Collaborative Software Initiative today announces the launch of the TriSano application and community. TriSano is an open source, citizen-focused infectious disease surveillance system that allows local, state and federal entities to collaborate for the good of public health.

With TriSano, the Collaborative Software Initiative provides a forum in which subject matter experts (i.e., doctors, nurses and epidemiologists) and software developers work together to facilitate the cre-ation of citizen-centric public health applications. This innovation ensures application features meet the specific requirements of each jurisdiction, allowing public health employees to achieve the goal of protecting lives.

See What is TriSano for name explanation. And say it with me ... its pronouced "tri-SAH-no" :)

Update Added the logo because I love it. Our talented UI dude Odanrot did it. That is Tornado spelled backwards (how cool is that?). Here is what it means:

The TriSano™ Eye represents a zooming (micro/macro) view on disease detection and prevention..

That is as close as I'm ever going to get to a bird of prey reference in a product. I love birds of prey. They have binoculars for eyes just like TriSano. Its true. Look it up.

JRuby 1.1.3 Performance

2008-07-21T20:40:00.000-07:00

I tweeted this, but thought I'd put it here too.

The JRuby guys have been busy. They released JRuby 1.3.3.

We upgraded to it from 1.1.1. Our app runs JRuby as does our build.

Check out the performance difference in the last few builds!

OSCON Postgres Day Postgres FTS JRuby Slides

2008-07-20T20:43:00.000-07:00

Ed Copony and I gave a presentation today at the OSCON PDX PUG Day.

Check out the slides Fighting Disease with PostgreSQL Full Text Search and JRuby on Rails

OSCON is here!

It may be the most epic OSCON ever. OSCON + Collaborative Software Initiative (CSI) co. meeting with open source luminaries in attendance + CSI nerd night - lots of stuff to do this week.

Good Day: R2 & Time Mag!

2008-07-03T15:31:00.000-07:00

So it is all a but of a blur, but I'm pretty sure today was a good day.

Two things happened:

Collaborative Software Initiative shipped R2 of our public health app
A couple hours before that, this Time Magazine article hit featuring CSI & the project.

Check the groovy pic of the Portland office peeps (too bad rest of team couldn't be there - pros and cons to a distributed company - mostly major pros):

It's nice to have a head of steam!

Stay tuned - its going to be a newsworthy summer.

And now I will partake in a hike in Forest Park because I'm spent.

Collaborative Software Initiative Job Posted

2008-06-03T14:40:00.000-07:00

Come join the public health core team.

See Core Developer Job Description.

See InfoQ Article on Tech we are using.

Update 25-JUN Note, we filled this position yesterday. We found an ideal person right in the JRuby community - JRuby experience, testing passion, and years of medical software dev.

InfoQ Article

2008-05-23T16:38:00.000-07:00

Check out the InfoQ article: Real world JRuby on Rails: Infectious disease reporting and management

Here is the intro:

JRuby and JRuby on Rails continue to being adopted by many projects. After Mingle, Oracle Mix or Sun's rewrite of mediacast.sun.com, a new project using JRuby on Rails has surfaced.

From the press release Collaborative Software Initiative (CSI), the company that brings like-minded organizations together to work on collaborative software at a fraction of the cost, today announced the release of the first open source, web-based infectious disease reporting and management system.

The disease reporting and management system, which is being piloted in Utah, will be adaptable in all 50 states and available under an open source license later this year. It is designed to support local health departments in the early detection and investigation of individual cases and local clusters of communicable diseases, while simultaneously meeting the state and federal needs of outbreak control, disease surveillance and epidemiologic research.

One thing to note further down in the press release: the project uses JRuby. We talked to Mike Herrick, Program Manager at the Collaborative Software Initiative, to talk about the experience of using JRuby for the project.

CSI Public Health Open Source Project Annouced

2008-05-19T06:49:00.000-07:00

I'm pretty proud to be involved in this project.

How often do you get to say this about the impact of your project?

When deployed, UT-NEDSS project will directly contribute to the prevention of sickness and death by effectively collecting, identifying, tracking and trending information gathered about infectious diseases and bioterrorism attacks. Additionally, this unique partnership is providing lessons for public health informatics on:

Open source software best practices

Lean Software Development practices

Building an open source community for a public health product

Contractual relations between public health entities and a commercial open source software company that support long-term product sustainability

We have a great team of folks working their hearts out on this. It truly is an honor to be part of the team.

I love moments like this - those strange joyous moments of clarity where anything seems possible in software.

Groovy New Commuter Bike

2008-05-06T20:06:00.000-07:00

I picked up a new commuter bike today to get me to work on my fairly incredible commute I used to call river walk. Its "river ride" now I suppose.

I got a 3 speed, pedal break simple old school euro bike. It's an Electra Amsterdam Classic.

I love my other bike, but it isn't practical for commuting.

Looking forward to a groovy summer on my new granny bike!

Hiring technique: "the blitz"

2008-04-19T07:21:00.000-07:00

So I like to think that I have become good at hiring people. I'm definitely not bad at it. Over the past couple years I have generally been good at hiring people who are way smarter than me. That's what you do when you are a Lean manager and you aren't stupid or insecure.

My techniques are not magic, but they seem to be quite different than how I see lots of managers/companies operating.

If you have determined that you truly need to hire someone, that generally means that your team is hurting - it needs resources that it doesn't have - at some level you are already failing. Your #1 mission as a Lean manager is to protect the team and to get it the stuff it needs to be successful. Once you are in this position, barring a crisis, you have no task that is more important than filling the position with the most qualified person you can find as quickly as possible.

So my co-worker Ed termed what I do yesterday, "the blitz". Here is generally how it works:

Step 1: Collect Qualified Peeps

Check your network for a trusted person.
If FAIL, post job to the places the kewl kids are. Examples here and here
If FAIL, post job to craigslist and batten down the hatches - you are going to get inundated with qualified and unqualified people contacting you. DO NOT put your phone number on these job postings.
If FAIL, do WHATEVER YOU NEED TO DO to find qualified candidates. Get creative, clocks tickn'.

Step 2: Quick Vetting

Pick the ones that look like they are truly talented and passionate.
Schedule in person interview for today or tomorrow
Short list.

Step 3. Homework Torture Test

Give the short list candidates a homework assignment. Something they can do in a couple of hours that will make it abundantly clear who is the most talented & the best fit.

Step 4. Second Opinion

Define decision making rules so everybody on the team understands how the decision will be made (e.g., unanimous vote, majority, or defer decision to manager for expediency and efficiency)
Depending on the position, pick one, more, or all people from the team to interview the finalists after they have reviewed the homework results for fit, talent, and passion.
Gather feedback

Step 5. Pick, yer done or Rinse and Repeat if FAIL

I think the only major difference compared to typical hiring practices is in how I do this. I go all in. It's a blitz. When I'm in this mode, I generally do it 8-12 hours a day until I'm done. I can generally wrap up hiring someone in <= 2 weeks from start to finish. And the results generally speak for themselves. This technique works great for me as the "hiring manager" and for job candidates.

Hiring people can be quick and painless or can drag on and on for weeks and months if you don't make it priority #1. While you have unfilled positions, you are continuing to fail (at some level). Your team is continuing to feel pain. Distraction grows and grows. Before you know it, your project is failing.

Anyway, this way works for me. Your mileage may vary.

Now next week I can get back to job #2 of a Lean manager: empowering the team/sharing leadership, staying out of the way, & adding real value off the critical path.

19-APR 9:30 PT: Updated with links to Lean SW Dev.

Web Application User Interface Designer

2008-04-14T12:59:00.000-07:00

We are looking for a part time User Interface Designer. Here is the description I came up with. Contact me if interested.

Looking for a paid part time contributor to one of CSI's projects. To start with up to 20 hours per week for 3 months paid hourly commensurate with experience. You will contribute by:

Producing wire frame mockups
Producing CSS, XHTML, images, etc.
Producing minimalistic Style Guide
Producing other design stuff
Interview SME (Subject Matter Expert) members of our project Core Team on their desires for the user interface
Making User Interface design decisions

We are looking for a freelancer and not a firm or an agency.

A little bit about you:

You were raised on Jakob Nielsen
You are pragmatic
You work well with distributed teams
You work well with application programmers (ours happen to be very nice and smart!)
You deliver
You are quick
You have a lot of experience with web application focused design
You have samples that you can show off
You have a long list of people who rave about you and your work

19-APR Update: this position has been filled via the blitz. Don't worry, we'll be back.

Core Team & Contributors

2008-04-12T09:14:00.000-07:00

I really like the concept of Core Teams. For Collaborative Software Initiative (CSI), our Core Teams are a mix of Subject Matter Experts (SMEs), and Developers. We create "whole" Core Teams & the team is jointly responsible for delivery. Its a true partnership model and its a beautiful thing to be a part of. Its the exact inverse of the typically adversarial relationship that often exists between SMEs and developers in more "traditional" development efforts.

We are still learning as we grow on the health care related project I am focused on. Our Core Team has had some success. The work load and what the team is capable of are both growing. We are becoming advanced enough together to take on more work and more people bit by bit.

We are currently experimenting with adding "Contributors" to the project. For any number of reasons, we are not able to add these people to the Core Team. But we have certain tasks that are well suited for part time contributors. It works well for us and the contributor. In some cases, the contributor just has an itch they want to scratch (e.g., want to learn Ruby/JRuby). In others we have a need we *need* scratched and we pay them to do the work.

I realize none of this is earth shattering - its common with open source projects. I am just saying that its exciting to be a part of it and see it working.

I am of course biased, but I continue to think that the CSI model is game changing. It certainly doesn't apply to everything, but it applies to an awful lot. Time will tell.

JRuby 1.1 Released

2008-04-05T08:10:00.000-07:00

JRuby 1.1 was released today.

We shipped our first release of an application Monday with the pre-release.

The JRuby guys are great. Its becoming a great little community. I'm happy to be a small part of it. There seem to be big plans for the future after 1.1.

After working with it for the past 3 months in anger, I continue to believe that its a very compelling platform.

Ruby.war

2008-03-27T18:23:00.000-07:00

I'm taking a break from the lead up to a release. I'm a little cooked and useless at the moment.

Pete and I worked on our deployment script last night. In the early days of our project, we thought we'd use Capistrano. Make no mistake, it is a great tool (I took it for a spin).

But when you are using JRuby, its a bit overkill.

I'm fairly new to the Ruby world, but I know the other deployment options for Ruby. They aren't awful, but they take a bit of work & have some moving parts to them.

A lot more work than handing over a .war file with a configured # of run-times to run within the JVM, a database script, and a runbook anyway. That in and of itself is compelling.

Oh yeah, you have to also run:

$apache-tomcat-6.0.14$ bin/startup.sh

(that's the runbook)

Ruby Authorization and Entitlement

2008-03-11T21:12:00.000-07:00

We are bearing down on our first release (first of many on a big roadmap). Times are crazy, yet a lot of fun. I feel strange posting here - its been a while. Well, at least I tweat fairly regularly.

Anyway, our final iteration of our initial release is primarily focused on authentication, authorization, and entitlement.

I have a feeling we'll end up rolling our own, but there are a number of Rails plug-ins out there for this purpose that may have some useful bits. Also, a number of Ruby based rules engines.

So far, my real tired google search turned up:

I'll dig into these tomorrow. Let me know if you have had success with these or any other plug-ins/gems that may be of use for a fairly sophisticated user authorization/entitlement impl.

I'll try to update this post with our findings.

19-MAR-2008 Update

We are still digging into this, but so far we haven't used any above given our situation, but Bill Katz Rails Authorization Plug-in looks somewhat promising so far. Seems easy enough to make work with SiteMinder in front of you.

27-MAR-2008 Update In case anyone stumbles upon this later, we ended up writing our own. Nothing against the other's we just didn't see enough compelling in them to take them on. Rolling our own seemed easiest.

Lean Software Development Joy

2008-02-15T20:39:00.000-08:00

As I mentioned recently, I'm freak busy at work. In a few months, things will calm down I'm sure and maybe things may pick back up here. On the other hand, things may get even busier.

In the meantime I have to say that I'm having a great time. Long hours to be sure, but the work is very rewarding.

For me, the most joyous part (other than working with exciting all FOSS technology, and being a member of a budding high performing team) is working with a business partner that truly seems to understand and embrace Lean Software Development. The proof will be in the eventual pudding, but its amazing what can be accomplished with a willing business partner that understands the fundamentals of software.

Or perhaps they just trust us, because we generally know what we are talking about. ;)

Or maybe we really are onto something and our business model is a better way to develop software.

Or maybe its all three :)

Mommydays

2008-02-15T20:19:00.000-08:00

My lovely wife Erin has for some reason become a blogger.

She used to make fun of me for blogging.

Now she makes fun of me for using twitter.

Anyway, you will quickly see that I married up if you read her blog. She talks about her more than full time job of being a mom.

On Architects

2008-02-02T11:36:00.000-08:00

Bill de hÓra excerpts from Dietrich Kappe.

I couldn't agree more with:

If you've made the transition from a hierarchical environment to an agile, self-organizing team, you know what I'm saying. You won't ever want to go back..

I've been called various things. My last title was "Architect". Before that "Technologist". And now "Program Manager". I used to be a pretty decent developer, but never a great one. I'm starting to get ok again. I'll be pretty good again soon. I've worked with a lot of great developers so I know the difference between good and great. I figured out somewhere along the way that I'm fairly good at sensing what teams need and sucking it up and doing the often crucial not-so-fun stuff. I like doing this stuff because it is rewarding knowing that I contributed to a team. As time allows I make technical contributions too (more and more now). There is nothing like being on a high performing team and carrying your own weight.

Seagull Architects are an abomination. I had a realization as I left my last job; for a variety of reasons I never want to have the title "Architect" again. In talking to other "Architects" and former "Architects" I was fairly surprised how common this sentiment is. I'd be happy with what I have now or "Developer". I also like Steve Vinoski's "Member of the Technical Staff". I don't really care what you call me, but I do care that I carry my own weight. More than anything I care that I'm on a whole team that is already or has the potential to become a high performing team.

Let's just all just stop using the term "Architect" in software. How about we get rid of "Engineer" too while we are at it.

Twittered

2008-02-01T07:48:00.000-08:00

I've been freak busy as of late. My blog is suffering and will likely continue to suffer. I'm going to give twitter a try. I've scoffed at it for a while, but am now succumbing.

You can follow me around here.

CSI on Linux Magazine’s Top 20 Companies to Watch in 2008

2008-01-11T10:12:00.000-08:00

Linux Magazine’s Top 20 Companies to Watch in 2008

Oh dear - 2008 is going to be busy.

Take the open source development methodology, mix well with enlightened self interest, stir with companies that have the dosh to fund serious development, and you have a really compelling business model. We're referring here, of course, to Collaborative Software Initiative (CSI).

CSI launched in early 2007, the brainchild of Stuart Cohen (formerly CEO of the Open Source Development Labs, which merged into the Linux Foundation in 2006). Its business model is to use the open development model to create necessary software for niche markets. For example, CSI's most recent project is creating software to automate the BITS Shared Assessment Program for companies in the financial services industry.
While it's unlikely that a community would spring up around this sort of software naturally, CSI is using the open development model to create software needed by industries and letting several companies split the development costs. Its role is to do the collaboration and community building, project management, tech support, and foundation building necessary to shepherd the software through its life cycle.

This is an idea that should have been obvious years ago, and we expect to see the CSI do well in 2008-- and attract copycat initiatives as well.

Update I liked the way it read so much that I put it in the post.

Hudson CI Rocks

2008-01-10T22:38:00.000-08:00

So I'm quite pleased with Hudson so far. Great FOSS Continuous Integration tool & community. Supports Java and basically anything else. And it has an IRC bot and who doesn't love IRC bots?

Fun in the JRuby Wiki: Warbler, Goldspike, & CI

2008-01-09T16:46:00.000-08:00

I'm having fun making minuscule contributions to the JRuby community. As I learn, I'm documenting stuff in the JRuby wiki. I'm really impressed with the people doing the real work on JRuby. Freak smart.

I keep running into Nick Sieger in my wiki work. He wrote a couple of very helpful things:

Warbler (replacing Goldspike Rake soon) & ci_reporter mentioned on the JRuby and Continuous Integration page I started in the JRuby wiki today).

My other favorite JRuby guy is Robert Egglestone. He fixed the Rails 2 Goldspike Servlet bug I found Sunday. He has recently updated the Goldspike Roadmap. A release of the Goldspike component is coming this month to follow up on the JRuby 1.1RC1 release the other day.

The discussion on CI caught Mike McKinney's eye. He put together a great how to on JRuby and Hudson.

I love open source and the internet machine.

Open Source Marching up the Vertical Stack

2008-01-07T21:29:00.000-08:00

Open Source is on the march up the stack - first operating systems, then integration software, and most recently commercial open source packages (CRM, ERP, BI, Content Management, etc.). Now applications? - where will it stop!?

My focus in working for the Collaborative Software Initiative (CSI) is on building commercial open source applications for industry verticals and government. I build on the shoulders of giants: Linux, Postgres, Java, Ruby, JRuby, and commercial open source packages.

At the application level, where is open source today and where is it going in industry verticals (government will be a different post as it is totally different)?

I see three categories *today*, but I bet over the next year I will see many more.

Regulated Software Today, there is a lot of desire to collaborate on systems that are mandated by the government. There are plenty of these in every industry. Pragmatic companies realize that there is absolutely no business value in differentiating here. It makes business sense to collaborate with their competitors to come up with an open, well-supported, cost effective solution to meet the regulation.

Industry Vertical Utility Applications Every industry has at least one industry standards body. These organizations do some great stuff. They also already have legal mechanisms in place to comply with anti-trust constraints. One common activity in industry standards bodies is putting together XML schema standards. My question for years has been: XML is nice and all, but how about we just sling some code, open source it, find somebody to support it, and solve the problem once and for all? I think we will see a lot of open source in this area in the coming years. XML is needed, but is just data exchange and only gets you so far. There is a common utility class of application that will be written in support of industries. I see this occurring more and more as there are success stories with the low hanging fruit in regulated software.

Industry Boat Lifters There are fundamental technical problems in many vertical industries. The cause can be many things from aging platforms to out of date industry networks to predatory vendors that create terrible industry inefficiencies due in part to the proprietary nature of their solutions. I see companies in vertical industries banding together to gently remove these types of inefficiencies. In some cases, their existence will depend on it. In others, the classic A rising tide lifts all boats argument will apply. Well placed open source industry vertical code could change the rules of the game in some industries.

Open source is just getting started in the vertical application space. We'll all see some very interesting things happen over the next few years. I think that what I described below is only the beginning.

BOOM BOOM BOOM SPLAT SPLAT SPLAT

2008-01-04T14:08:00.000-08:00

I've been reading to my daughter a lot lately.

She is kinda past Mr Brown Can M00! Can You?, but when I read my new colleague, Pete Lacey's recent blog post, I started chanting it in my mind:

BOOM BOOM BOOM

Mr. Brown is a wonder.
BOOM BOOM BOOM
Mr. Brown makes thunder!
He makes ligtning
SPLATT SPLATT SPLATT
and it's very, very hard

to make a noise like that!

So Pete & I will be riding the CSI Rocket together. Wish us luck. I'm extremely excited.

PS: What Pete said about the power of the interweb.

JRuby .war packaging for Rails 2.0

2008-01-01T19:30:00.000-08:00

I have been hacking on JRuby a bit lately.

I set out to deploy a Ruby on Rails 2.0 application to Tomcat 6 on Monday and it was harder than it should have been.

Some of my angst was no doubt the fact that I'm a newbie, but in case it helps anyone, here are the key bits that made it work for me.

I started with using the Goldspike .war generator. It worked, but was a little clunky.

So then I tried Nick Sieger's Warbler which is much simpler & has more of a Ruby feel to it. I found myself down a rat hole, however in that the Warbler gem comes with JRuby 1.0.1 and that doesn't seem to work with Rails 2.0. If you try to run on it you will be greeted with something like:

INFO: Deploying web application archive your-app-name-here.war 
Exception in thread "ObjectPoolManager" java.lang.NoSuchMethodError: org.jruby.RubyHash.op_aset(Lorg/jruby/runtime/builtin/IRubyObject;Lorg/jruby/runtime/builtin/IRub
yObject;)Lorg/jruby/runtime/builtin/IRubyObject;
        at org.jruby.ext.openssl.X509Name.createX509Name(X509Name.java:98)
        at org.jruby.ext.openssl.X509.createX509(X509.java:40)

The documentation is a little bit sparse on this topic & I didn't stumble upon anything in my googling around. The fix is simple - you just have to use a compliant version of JRuby. 1.0.3 works. Here are the steps I took to fix it:

Download jruby-complete-1.0.3.jar
Replace the JRuby that Warbler uses - one approach is to figure out where Warbler lives (
```
gem which warbler OR jruby -S gem which warbler
```
) and replace the default jruby (1.0.1) with 1.0.3

warbler war:debug prints a YAML dump of the current configuration

is definitely useful in tracking stuff down.

Anyway, Warbler is quite simple & useful as it should be.

What's not to like about:

gem install warbler 
warbler war 
warbler config
warbler war:clean

and warble.rb (config file)?

Note that you can also run it directly under JRuby:

jruby -S gem install warbler
jruby warbler war
jruby warbler config
jruby warbler:clean

Warbler is very tidy. It put's all the generated bits in the tmp dir of your Rails app. This is opposed to Goldspike which puts it all in the WEB-INF dir in the root of your app.

Update I put some of this and more in the JRuby Wiki.

Go Flyers

2008-01-01T09:52:00.000-08:00

Sarge pointed out to me that my Dayton Flyers are now nationally ranked in the top 25.

I get pretty fired up about them once they are good. I am the epitome of the fair weather fan. I have cried in my beer many times over the Flyers. When they are good it generally goes like this:

Good regular season
Couple 2-3 clutch wins
Split with Xavier
Make the NCAA tournament
Lose in the 1st game to Purdue or someone by 1 point on a b.s. call by some ref

Anyway, I'm paying attention now.

Go Flyers!

2007 Predictions Revisited

2008-01-01T08:11:00.000-08:00

Before I make my 2008 predictions, I'm going to revisit my 2007 Predictions.

My predictions from last year with my scoring & commentary:

CORRECT This really did happen, wow.
Even the day coders will run screaming from WS-*. The battle is won; we night coders killed it because we didn't want to be controlled by vendor-pires and it is fundamentally flawed technology. Sadly, even though I believe that it is dead, it will take 2 more years before the industry echo chamber comes to terms with it. Software vendors who are heavily invested in WS-* will spend 2007 doing two things: one last gasp at making WS-* happen and quietly writing their Plan B MRDs.
CORRECT 2008 will start to change this one I think.
REST usage will increase in lieu of WS-*, but it won't unseat messaging and other middleware.
PARTIAL CREDIT There were some pretty good spaz outs on this one. Adobe Flex, Silver Light, etc. I have to think about this again for 2008. Ajax is a pain, but I really do mutter "What's with that loading ..? when I use a Flex based app.
People will settle down a tad about Ajax. It's great and all, but I've been seeing more and more botched impls. Like just about everything, its just a tool not a dogma.
CORRECT Blasted XML.
XML will finally be considered one tool in the tool chest just like every other technology. Development teams will only use XML where it actually adds value and not force it into places where it does not belong.
PARTIAL CREDIT It is growing, but it is complex. I prefer bare metal. Running VMWare for a 1.5 years taught me a lot about this. I run bare metal Linux now on my primary workstation and it is way better. Virtualization is neat and all, but for my money, commodity hw is the way to go. Simple, see.
The virtualization march will continue.
PARTIAL CREDIT Lots of yapping, Oracle bought Tangosol, etc.
Distributed Cache and JavaSpace usage will accelerate as more people grok the power of this architecture style.
WRONG Maybe this year. I learned heaps working with Jini/JavaSpaces. It changed the way I look at distributed systems forever. Maybe Rinda has a shot.
Apache River will breathe new life into Jini/JavaSpaces. Both will see lots of new interest and implementations. 2007 will be a "rebuilding year" (as they say in sports) - 2008 will be the big year for a Jini/JavaSpaces come back. Someone from Sun will formally apologize to all the people maimed by J2EE in 2009 and acknowledge that they should have marketed Jini as a service technology from the beginning (ok just kidding that isn't going to happen).
PARTIAL CREDIT Big hat no cattle.
The Open Source patent war will begin in earnest.
CORRECT Funny how this all works out. It certainly is on track. CSI (my employer) and others are helping this along. More on this in my predictions for 2008 ...
Community Source Software will slowly start to take off in industry verticals as more executives grok the possibilities and come to terms with the fact that they are already sharing industry vertical software with their competitors; they just don't have access or any control of the source code. Look for a success or two in 2007 which will set the stage for a pandemic by 2009.
PARTIAL CREDIT
Even more smart people will start blogs or at least start reading blogs which result in even more transparency and open collaboration between software vendors, customers, and consultants and ultimately better software, more innovation, and less waste.

Fuzzy Egg Sandwich

2007-12-30T07:56:00.000-08:00

I have somehow become a legend in my home for my egg sandwiches. My belated Christmas present to my readers is the recipe. You of course will not be able to replicate exactly, but you can try ;)

One pad of butter to the frying pan (lowish heat)
One organic egg cracked open to the frying pan (sunny side up) - cover
2 pieces of Dave's Killer Bread to the toaster
When toast is done, egg is generally done. Turn off heat to egg
One slice of Horizon Organic Cheddar Cheese to the top of the egg - cover
As the cheese is melting (1 minute), start the toaster again to maintain crucial bread-heat-ratio
Remove egg and place on one piece of toast
Place ample amounts of hot sauce (either Marie Sharp's Habanero or Bali Spice Hot Chile Sauce) on the other side of toast
Put hot sauce side on top of the egg side
Slice in half
Hand out to your family / devour
Bask in the glory of your egg-sandwich-making-prowess

Remember the Milk = Neat

2007-12-24T05:51:00.000-08:00

People who have worked with me may remember me getting all excited about various task management approaches. I liked Getting Things Done and other techniques before that.

I typically go through phases of being very organized & very disorganized as I'm in various states of work flow/learning levels. Well I'm in an organized state right now as I have a lot to do for CSI and am learning at a high rate (Ruby, JRuby, the Rails way, etc).

I saw on somebody's del.icio.us a link to Remember the Milk. I remember hearing about it from Jive's Bill Lynch in the spring.

I gave it a try the other day and am pretty happy with it so far.

Here is why:

You can have as many lists as you want. Categories don't work for me.
It can IM you when things are due
It has an offline mode with Google Gears
It works well with my phone
You can email it tasks - for instance I was on a plane yesterday and I mailed it like 20 tasks. I categorized them today
You can render it by URL (like Evolution's task manager)
Very searchable
Integrates your task lists into views across them well

Time will tell if I stick with it, but so far so good on Remember the Milk.

Update 30-DEC

I forgot to mention perhaps the best part: Keyboard Shortcuts.

Changing the rules of enterprise software

2007-12-20T07:18:00.000-08:00

I had a good conversation with Matt Asay yesterday about open source moving up the stack & CSI. I got to know him a bit over the last few years via blogs, OSCON, OSBC, & Alfresco. He's a lot of fun to talk to about open source & everything else.

2008 is going to be an exciting year.

Starting to think about 2008 Predictions

2007-12-16T06:04:00.000-08:00

I had fun making some predictions last year.

I plan to do the same thing this year in a week or so.

It is a bit funny looking back at what I thought for 2007.

I was spot on for some of it, but way off for others. I voted with my feet with one of them ;)

Recruiting == Lots of Work

2007-12-13T19:25:00.000-08:00

One of the things that I like to think I'm good at is putting together lethal software teams.

Whenever I have had the opportunity to build a team, it consumes me entirely until the team is in place. My logic is you can't do jack until you have the team. Once you identify the need for people, there is no priority that is more important than getting the best people you can find.

I think I'm close to being done for this round, but I suspect we'll be doing a lot of hiring in the future.

Stay tuned.

I hope that this round is nearly over. Although I like putting together teams, it exhausts me.

JRuby Book

2007-12-11T17:18:00.000-08:00

I read Practical JRuby on Rails Web 2.0 Projects: Bringing Ruby on Rails to Java over the past couple 2-3 days.

It is a pretty good book. It was released in September and is already a bit out of date. JRuby & the Ruby world in general seem to move pretty fast.

The topics I liked the best were:

Nuts & bolts of Java & JRuby integration
Talk of REXML limitations and the (nice) option to just bang out to the multitude of Java XML APIs should you need extra features
JRuby & JMX
JRuby & MOM (Message Oriented Middleware) - ActiveMessaging

After reading it and messing with JRuby a bit, I'm more certain of what I thought in May (ok I didn't say that exactly & I've lost track of Scala - focus on the JVM :) ).

There clearly is a very bright future for Ruby in general & in particular Ruby on the JVM.

What (obviously) makes JRuby + Java so compelling is:

There is a Java API for *everything*. I hope to stay in Ruby code as much as possible, but it sure is nice knowing that the APIs I have been using for the past 10 years are easily available
The majority of companies run Java in their data center & are very comfortable deploying to it. Many are also not comfortable adding platforms (e.g., C Ruby)
The JVM has had a tremendous amount of engineering put into it
Ruby is a fantastic language that truly is a joy to work with
Ruby has heaps of momentum & enthusiasm behind it

Job Opportunity

2007-12-08T12:10:00.000-08:00

Please contact me if you are interested & have the skills (mike at-sign csinitiative dot com:

Collaborative Software Initiative

Web application developer(s) to join an all-star team building a high quality, commercial open source, platform neutral, public health application utilizing state-of-the-art FOSS tools. This is an opportunity for a self-starting individual to participate in a ground-breaking effort that combines non-technical subject matter experts with skilled professional developers in a major, well-funded, open source community and project.

Job Details

Reports to: Project Program Manager

* Compensation: Salary & Benefits or Contract

* Location: North America work from home (if out of Portland, OR) with occasional US travel

Responsibilities

* Develop high quality test cases and application code

* Contribute to release planning, iteration planning, and retrospectives

* Communicate clearly via strong oral and written communication skills

Technical Skills

Required

* Ruby

* Java

* Testing frameworks

* Relational databases (Postgres experience preferred)

* XHTML

* Javascript

* CSS

* XML

* Ajax

* Linux

Desired

* JRuby

* RESTful Web Services

* Java GUI / Java Web Start / Eclipse RCP

* Systems integration

Methodology

* Open Source Software Development Best Practices

* Lean Software Development / Agile

* Test Driven Development

Update 11-DEC-2007 I got several questions on preference of Ruby vs. Python. Ruby is the preference. To make that clear I removed Python and Jython. Sorry for the confusion. I also added systems integration as there will be a fair amount of that as well.

Rails 2.0 - gem install actionwebservice

2007-12-07T15:32:00.000-08:00

I'm kicking it in Sunriver before starting full time at Collaborative Software Initiative Monday.

Rails 2.0 is out.

From the link:

ActionWebService out, ActiveResource in

It’ll probably come as no surprise that Rails has picked a side in the SOAP vs REST debate. Unless you absolutely have to use SOAP for integration purposes, we strongly discourage you from doing so. As a naturally extension of that, we’ve pulled ActionWebService from the default bundle. It’s only a gem install actionwebservice away, but it sends an important message none the less.

Take that WS-* !!

Next Chapter

2007-11-27T22:44:00.000-08:00

I am going to start the next chapter of my career at Collaborative Software Initiative.

As I get my feet under me there in the coming weeks I'll say more about it.

I have had a fantastic time in my current position & have learned a ton. I have worked with wonderful & brilliant people. Leaving is definitely bittersweet.

But this opportunity is where my heart is. I am passionate about open source and more specifically open source in vertical markets & government.

You only live once & you only get a shot at fulfilling your dreams a couple times. When opportunity knocks you must jump with both feet.

Wish me luck.

Ubuntu Thinkpad

2007-11-26T23:17:00.000-08:00

I spent part of my evening installing Ubuntu on my new Thinkpad.

The download of Ubuntu went fine. Burning a CD/DVD took 6 tries!

After that, though, it was smooth sailing. Ubuntu found my wireless network & prompted me to install the proprietary NVIDIA driver for my graphics card.

Yay.

Disconnected Client Alternatives

2007-11-23T09:52:00.000-08:00

I want to build a *simple* cross platform fat client application. It will allow users to enter data while disconnected and then upload it to a server when they have a connection.

I'd prefer not to use Java (black magic) or AIR (not ready?).

I'd prefer to hurt the web as little as possible (i.e., the more web technology in it the better for skills transfer)

I'd prefer it to be open source.

I'd prefer to not have to hire specialist ninjas to be able to build it. Every day web programmers should be able to get up to speed with it.

What are some good choices?

XUL? Mozilla Prism (some day)? Google Gears?

Update - Nov 27

This post got some helpful comments - thanks! And a nice post from Erik Onnen - thanks! There is much love for AIR. I went to the Adobe Bus Tour and generally like Adobe. The proprietary nature of it is tough pill to swallow - opening Flex was a step in the right direction, but I'd like to see more more more.

Release It!

2007-11-23T08:03:00.000-08:00

What Steve Vinoski said:

Release It! is truly excellent.

REST UML Drawing

2007-11-19T16:56:00.000-08:00

Nice REST UML drawring style by Benjamin Carlyle.

Kenny and Zukes

2007-11-18T12:05:00.000-08:00

I went to Kenny and Zukes yesterday. I had a pastrami sandwich. It was freakishly good.

Erik Onnen claims it is better than NYC. I have to (gasp) agree. Dang good. And right down the street from me. Yum.

RESTful Web Apps

2007-11-16T05:15:00.000-08:00

Nice post by Matt Raible on RESTful Struts. One thing Matt, code gen is a bug ;)

I have been thinking about this for a couple 2-3 weeks. How do you build apps that are apps and RESTful services vs. building an app and then a RESTful service bolt-on-side-car?

Pete Lacey had a nice demo with a Ruby on Rails demo of this type of thing last week.

I find it really entertaining that more and more people are having the REST light bulb go off in their heads. It is like - wait! I have been hurtn' the web for how many years!?!? You mean it could be this simple and I can have all this reliable, scalable, security booya because the web works so well? Holy s%*#!!!!

Java based Atompub Options

2007-11-15T06:51:00.000-08:00

I know of the following Java based Atompub impls:

The Restlet folks have also talked of implementing basic Atompub support which is appealing.

Am I missing any other impls?

Anybody have any happy experience with these or other impls?

We looked at several a couple 2-3 months ago, but all have changed since then.

QCon REST

2007-11-09T12:40:00.000-08:00

I had a great time at QCon Connecting SOA and the Web: How much REST do we need?.

The presentations / discussions were great. I learned a couple new things. It was definitely the best conference I have been to in a long time.

Having dinner with Dan Diephouse, Stefan Tilkov, Steve Vinoski, Sanjiva Weerawarana, Pete Lacey, Glen Daniels, Jim Webber & Patrick Logan was also a lot of fun. I have known many of them virtually for a long time. It was nice to talk in person and share some great laughs.

Stefan Tilkov - REST Talk

2007-11-04T06:15:00.000-08:00

Good stuff.

Ducks

2007-11-04T05:33:00.000-08:00

For whatever reason I stopped watching sports when I moved to Oregon 6 years ago. I think it was because I started getting into my own sports more here (fly fishing, hiking, biking) & it was hard to see the teams I liked on TV here.

I have followed Oregon State and Oregon a little bit, but not very much.

Yesterday was the first time I felt myself becoming a Duck Fan.

My dad called me from the Michigan - Michigan State game to report that Michigan State was coming back - I was watching the Oregon game completely unaware. I grew up a fairly big Michigan State fan.

I think that the fact that Oregon is doing so well is a big part of my interest. But as I make more and more Oregonian friends (who are fanatics for Oregon), I find myself caring a bit more. It is funny how the longer you live somewhere the more it feels like home.

That Dixon dude is ridiculous - very exciting QB to watch. I hope his knee isn't canned.

Update PS. I am still very sad that Michigan beat Michigan State. I hate it when MSU loses to Michigan or Notre Dame.

PM

2007-10-30T07:17:00.000-07:00

Good stuff on Project Managers & Project Administrators by Dave Christiansen. He links to Are you a leader or a tracker? by Scott Berkun.

See also Dave's post on The Meaning of Rigor in Project Management

28-OCT-2008 Atompub Server Count Over Under

2007-10-28T15:13:00.000-07:00

How many open source & proprietary Atompub servers will be available October 28, 2008?

If I was the house and I was setting the over-under, I'd say 10 & 6 respectively.

Update

Yikes: James Snell: We (IBM) probably have at least that many already in production within our firewall with more on the way.

I'm glad this is make believe over-under.

To clarify, I meant general purpose - integration focused Atompub servers. I predict that Atompub will result in 10 OSS and 6 proprietary general purpose - integration focused - Atompub servers. I'll go ahead and set the "Atompub built into your existing commercial product/service (e.g., Lotus Connections)" at 350.

Atompop

2007-10-28T14:50:00.000-07:00

Bill de hÓra talked about a potential Atompop RFC a month ago.

He said: Messaging is a big deal and having something run over Atompub would justify an RFC.

+1 to that

Otherwise a lot of people are more or less going to reinvent it anyway.

More and more I'm seeing HTTP as the preferred protocol and protocol gateways to messaging. As we are using a bit of messaging and a bit of Atompub, it would be nice to have an RFC that would define how to do this one way. It might not be necessary (or achievable), but it could also be useful to include how to go from messaging to Atompub (i.e., the messaging side). I mean specify the "server-side gateway" & the "client-side gateway".

HTTP

2007-10-28T09:13:00.000-07:00

I first started to really know HTTP when I was the technical lead of a Netegrity SiteMinder implementation in 2001. Prior to that, I just built web applications on top of it & didn't really know a ton of details about what was really happening on the wire.

I ran into so many bizarre problems with obscure web containers that I was forced to do all sorts of HTTP traces, repeatedly read the specs, etc. A memorable problem was a version of Chilisoft truncating SiteMinder's ginormous SMSESSION cookie. Identifying that defect took some work. Sun was awesome to deal with though - I remember them getting us a patch in 2 days.

Anyway, its an older book, but I'm reading HTTP: The Definitive Guide. I am surprised at how much I still don't know about HTTP. Anyway, I really like the book - it makes me like HTTP even more.

Enterprise 2.1

2007-10-28T08:13:00.000-07:00

Mark Masterson is working on consolidating some enterprise memes into a list called Enterprise 2.1.

The whole naming thing is a little silly (e.g., Web 2.0, Enterprise 2.0), but I suppose it does have its place in the communication of ideas.

Anyway, here is the list:

The impact of social software on the enterprise (the original Enterprise 2.0 meme)

The superiority of ROA over other distributed system design styles

The need to provide for reflective, adaptable behavior of business users rather than lock them into static, predefined processes

The desire to reduce (and control) complexity and system entropy through the rigorous application of ideas like lesscode, convention over configuration, and favoring simple, lightweight solutions over large system designs.

It might not fit - or might be too much of "the how" rather than "the what", but something about open standards and open code on the list might be appropriate.

Creswell on Architecture

2007-10-26T19:18:00.000-07:00

Dan Creswell says stuff about architecture: Consider how prevalent the use of frameworks is within our industry and think about the fact that in many cases one simply writes a POJO or two and leaves the rest to the framework. The framework makes life easier, it solves the big problems but it also exerts force on the design of our software as after all we must write it to follow the appropriate conventions, implement the appropriate methods etc.

The very worst example of the framework trend is seen in the decision to purchase a mammoth framework offering that provides everything in one box as an “integrated solution”. A huge stack that gets connected into everything and exerts massive gravity on our architecture. Everything becomes an exercise in warping aspects of our system to fit with this stack and the assumptions of its creators. Essentially we’ve bought “architecture in a box”.

The older I get, the simpler I like things. I don't have anything against frameworks, but they often become massive technical debt by themselves.

Einstein is over quoted, but more and more my views on software architecture are centered on Make everything as simple as possible, but not simpler.

This means writing the tiniest amount of code you can. For instance, use HTTP clients, use JMS clients, write a security impl., perhaps a simple feed reader. Deploy some really good web architecture. Some good monitoring foo. And then build in unit, integration, and acceptance testing from the beginning. And yer done. At least if you want to take advantage of all the engineering that already went into the web - which at this point is kindove lot.

Google Reader

2007-10-24T21:32:00.000-07:00

I started using Google Reader a couple 2-3 weeks ago rather than Bloglines.

It is darn-tootn' good. And you can just export your feeds from Bloglines and import into Google Reader. 2 minutes, done.

Thanks for the good year (or so) Bloglines - you were very good to me . . .

Wake up folks - architecture is YOUR responsibility

2007-10-24T19:50:00.000-07:00

If you want to make SOA work in a heterogeneous environment, why would you want to limit yourself to one technology, one approach? You’re buying right into their strategy of locking you into a platform. That’s only good if you sell platforms. Wake up folks - architecture is YOUR responsibility, not that of some vendors hawking middleware!

Ron Schmelzer of ZapThink

Simple

2007-10-24T06:34:00.000-07:00

I descended into a simple blog link vortex - a couple notables:

Bill de hÓra: All successful large systems were successful small systems

Dare Obasanjo: My Website is Bigger Than Your Enterprise

Ryan Tomayko

Sam Ruby: Radical Simplification

Proposed Standard

2007-10-24T05:58:00.000-07:00

Oh just a proposed standard huh?

So what does "Proposed Standard" mean? That might not sound very stable - "Proposed" - but that term does have special meaning in the IETF. The best way to explain it is that there are plenty of other protocols that you use every day that are "Proposed Standards", for example: WebDAV, TLS, LDAP, SMTP, SIP, and IMAP. Of course, if you want to know more about the IETF's standards process, they have it documented, in RFC 2026

Joe Gregorio talks (now at Google) about GData and how says: This is great news for us because the Google Data APIs are built on the Atom Publishing Protocol. The current Google Data APIs are based on early versions of the AtomPub specification and now that AtomPub is a Proposed Standard work will begin on getting all of our Google Data APIs compliant with RFC 5023.

Push & Pull Naming Standard & the Uniform Interface

2007-10-23T19:13:00.000-07:00

The uniform interface is great. Resource name + GET, POST, PUT, DELETE & you are done. This is an amazingly simple / powerful constraint of the REST architectural style.

I spent a little time today thinking through a naming standard for our integration architecture. We need to support Push & Pull. This is essentially REST/AtomPub & Messaging.

For REST, I liked what RESTful Web Services had to say:

. . . So which is it? URI as UI, or URI opacity? For once in this book I'm going to give you the cop-out answer: it depends. It depends on which is worse,for your clients: a URI that has no visible relationship to the resource it names, or a URI that breaks when its resource state changes. I almost always come down on the side of URI as UI, but that's just my opinion.

It references URL as UI (Jakob Nielsen) and Universal Resource Identifiers -- Axioms of Web Architecture (Tim Berners-Lee)

URI Templates seem like a natural fit for specifying URI standards & URIs that are in use.

What about on the Push side (i.e., traditional MOM/XMPP) - what is the analogue? Is there one?

I toyed with the idea of using the same verbs (GET & POST at least), but I think that the semantics are different enough with messaging that it is either too clever or just lame to use a subset of the same verbs. The main issue is that GET isn't idempotent in messaging. I guess you could use GET for Browse, DELETE to pop off a queue, and POST to send a message, but it just seems too contrived to me. These are different styles - why break people's head with foolish consistency?

In the past I have had good luck with NOUN.VERB in messaging which is similar to REST in a way if you standardize on it.

Other ideas are NOUN.INPUT, NOUN.OUTPUT; NOUN.REQUEST, NOUN.RESPONSE.

Any other ideas / experiences?

Sweet Slides: The Web: Distributed Objects Realized!

2007-10-22T20:48:00.000-07:00

OOPSLA 2007 The Web: Distributed Objects Realized! (Stu Charlton and Mark Baker)

Good paper(s) too:

REST is clearly not the "end of history" with regards to network-based software architecture. At smaller scales, with somewhat different desirable properties, other styles, such as Remote Data Access, or Event Based Integration, continue to solve important organizational and technological challenges. Beyond the current iteration of the Web, future requirements will require a re-evaluation of REST's constraints and desirable properties for a new set of requirements. This may lead to relaxing some of REST's constraints, or an introduction of new constraints (Extending REST for Decentralized Systems). Regardless of the direction this may take, a continued means to successful future systems architecture will be the discipline of objectively evaluating constraints, and the properties they induce, for the new generation of global scale, networked-based software systems.

Via Stefan Tilkov

Update fixed broken link to presentation. Also added this image as it caught my eye:

I distinctly remember the "Object Web" phase of distributed computing. I remember being a participant in it. I bought into CORBA. I remember saying things in 1998 like, "we need more - we can't be constrained to HTTP GET and POST in the browser. We need to be able to bust a socket and talk to an ORB for a richer experience" I was kinda clueless then. At least I had a lot of company - everybody including Lotus Domino which I was integrating with when I made that comment was busily adding CORBA capabilities to there whos-its and whats-its. Well everybody besides MSFT and friends, but that is a different story.

Almost 10 years later, we still just have GET and POST in the browser, but XHTML 5 will eventually fix that. It turns out you can get pretty far with just GET and POST. You can get even farther by just discarding distributed object / RPC technology all together and using the REST uniform interface.

QCon - Connecting SOA and the Web: How much REST do we need?

2007-10-22T20:13:00.000-07:00

Like Patrick, I'm very excited to head to SF for QCon.

It will be great to meet Stefan Tilkov, Steve Vinoski, Pete Lacey, and Dan Diephouse in person. We'll no doubt have a lot to discuss.

History of HTTP

2007-10-20T17:48:00.000-07:00

While at Powell's Technical this morning, I picked up HTTP The Definitive Guide.

It has a couple of good links on the history of HTTP:

Why a new protocol? Tim Berners-Lee (1991)

A Little History of the World Wide Web

Web Architecture from 50,000 feet

What the software of the future looks like

2007-10-20T08:12:00.000-07:00

A nice post from Jacob Kaplan-Moss:

Let me try to explain what I’m talking about. You want an API without having to write a line of code? It’s called curl, and it ships with your MacBook. And just look at how simple the APIs are in your favorite language.

The wins go deeper than APIs, though. Think about what it would take to add load-balancing to CouchDB. I’ll give you a hint: perlbal. Or what about adding a transparent caching layer? Try Squid.

HTTP is the lingua franca of our age; if you speak HTTP, it opens up all sorts of doors. There’s something almost subversive about CouchDB; it’s completely language-, platform-, and OS-agnostic.

Look, CouchDB may succeeded, and it may fail; who knows. I’m sure of one thing, though — this is what the software of the future looks like.

Patrick and I were talking about this a bit yesterday. It is tough for some in the middleware world to get this. It will come with time. It is a bit nuts that your integration architecture really is just the judicious use of HTTP, well designed URIs, and data formats. How that architecture is realized (CouchDB, Squid, perlbal, etc. etc.) really won't matter over time.

If you can't answer yes, yes, and yes to: completely language-, platform-, and OS-agnostic when it comes to your integration architecture, you are doing something very wrong.

Via Simon Willison

Ubuntu 7.10 Install Today + Garden Shop + Semantic Web

2007-10-20T06:06:00.000-07:00

I am going to do a couple 2-3 things today:

Install Ubuntu 7.10 (Gutsy Gibbon) Check out the screen shots
Go to the garden shop and get some ferns
Buy & start to read one of these if they are not at Quimby Warehouse (code for not in stock at Powell's Technical down the street)

XMPP Slides

2007-10-17T18:47:00.000-07:00

Check out the nice slide deck on Peter Saint-Andre's blog.

Bloatware vs. Pairing Knife

2007-10-14T12:25:00.000-07:00

An interesting post from Eric Newcomer on the HPTS (High Performance Transaction System) Workshop:

For example, one of the Web guys commented that they did not want to have to buy the whole set (i.e. complete product) when all they really needed was a paring knife, yet the only choice they are offered is to buy the whole set.

. . .

"Bloatware" is definitely part of the issue. Ironically so are many of the things a lot of us have worked to define, create, and promote during the past two or three decades around guarantees of atomicity and isolation. None of the Web companies use distributed two-phase commit, for example. They only use it to deque an element and update a database in the same local transaction. So much for all that work on WS-TX! ;-)

. . .

It will be very interesting to observe over the next few years the extent to which the ideas and techniques in the custom built solutions become more widely adopted and incorporated into commercial products. One of the inevitable questions, as raised during the discussions, is how broad the market is for such things as Google's file system and big table, or Amazon's S3 and Dynamo.

The next few years will indeed be interesting as we enter the post WS-* era. It will be interesting to see which middleware companies survive. For one I wonder if Google & Amazon will more completely enter the enterprise integration business by productizing some of their core technology.

PDX Software

2007-10-14T09:34:00.000-07:00

The Silicon Forest blog has a brief interview with Jim Zemlin (The Linux Foundation - formally OSDL):

You have to look at the open source movement as a global phenomenon. My day literally starts with a call from Moscow and ends with a call from Beijing. From that perspective, geography is less important in this incredible global collaboration over the Internet.

Having said that, I'd like to look specifically at Oregon and where I believe Oregon provides a leading role...starting with Linus...I think it's important to Oregon that Linus lives there. I think that is a real indication of the importance of Portland as a place where real leaders in the open source community reside and do their work.

. . .

There is opportunity for places like Portland because of this fact, that it is a global phenomenon, to really attract top people to the area, either to be close to their peer group, or just based on the fact that it's a more affordable and superior standard of living.

It will be interesting to see what Portland looks like in 20 years. Come to think of it, it will be interesting to see what lots of places look like in 20 years. I am naturally most interested in Portland since I will in all likelihood still be here involved in the creation of software.

I see more small - mid sized software companies in Portland's future. Also small satellite development centers for mid-large companies as the world continues to flatten. I don't see many major companies moving here or anything like that. Portland has a decent amount of talent in open source & software more broadly.

The biggest thing Portland really has going for it is people (especially creative people) will typically move here in a heart beat as it is such a great / unique place to live.

I don't see Portland ever being a Tier 1 city or major tech hub. But clearly over time (20 years or so) SF, Portland, and Seattle will continue to morph together. I see a very bright future for software in Portland.

Distributed Scrum

2007-10-13T15:41:00.000-07:00

InfoQ has a nice presentation:Planning and Maintaining the Rhythm of Distributed Scrum.

Summary At Agile2007 we heard the tale of a distributed Scrum project with 50 people in 4 continents. BMC Identity Management decided to build their next generation product, including architectural changes and component integration, using Scrum to handle the uncertainty of their product's requirements.

Will the technical debt consume us?

2007-10-11T18:58:00.000-07:00

Pete Lacey quotes Anne Thomas Manes:

The problem is caused by the root culture of IT — project-driven funding models, a cobbler’s kids perspective on investing in infrastructure that helps IT (rather than a particular project), and a propensity to never decommission applications. IT systems have grown organically for the last 40 years. They’re a mess. It requires a fundamental change in the way IT operates as a service provider within the organization.

Here is the post on the Yahoo service-oriented-architecture list.

She is of course correct.

And all the while the technical debt is piling up. Maintenance is accounting for more and more of the IT budget. Soon there will be nothing left for new development.

There are of course solutions to this problem, but they require a lot of discipline and vision.

I think the discipline and vision are much less costly & more effective than the annual madness of budgeting in your typical large company.

Update 12-OCT-2007

I don't know why I didn't think of this last night, but Patrick quoted 12 Questions with Mary Poppendieck just yesterday in our internal wiki:

The first step in moving from forecast-driven projects to feedback-driven... is to change the measurements. The book "Rebirth of American Industry" by William Waddell and Norman Bodek makes a good case that the measurements imposed by traditional cost-accounting methods are the biggest impediment to the successful implementation of lean manufacturing. Similarly, I believe that the measurements imposed by traditional project management methods are the biggest impediment to the successful implementation of lean development. In particular, instead of measuring variation from plan, we need to start measuring the delivery of realized business value.

AtomPub RFC - RFC 5023

2007-10-08T18:29:00.000-07:00

James Snell notes that AtomPub is now RFC 5023.

I think that AtomPub has heaps of potential.

Ubuntu 7.10

2007-10-07T07:14:00.000-07:00

It's coming soon.. 11 days.

Ubuntu rocks. The new version is going to be interesting to watch. Ubuntu is already fantastic. But it seems like this could be a break out release with all they eye/feature candy.

ESB Spaz Out Excerpts

2007-10-06T10:29:00.000-07:00

A couple 2-3 excerpts from the ESB spaz out that caught my eye. I have largely sat this one out since playing a small part in starting it. Perhaps I'll say something soon, but for now, here are a few quotes that caught my eye. I'm too spent from a long week to say much more.

Steve Vinoski: Reactions to the ESB Question

Eric Newcomer comment: I tend to think of the IT world as divided between systems designed before the Web, and those designed to include it. The mindsets are very different, as you point out, but I would add that the pre-Web mindset is kind of driven by mainframe centric designs - I like to think that the issues with existing middleware (and perhaps binary languages) is due to the fact we always felt we had to design in all the features/functions of mainframe based systems in order to entice enterprises to move those apps to standards based systems.

There is a lot of truth to this. I live with a lot of big iron. Interestingly, AtomPub & the whole pull based feed approach to integration is somewhat similar to batch processing. Perhaps it can help bridge that gap a little bit? Hey, that would be cool - Atom feeds from VSAM files & AtomPub from CICS. Maybe, but it is the batch cycle itself that has a strangle hold. Batch cycles that have been nurtured for 30+ years are very difficult to untangle. Things are really bound up there.

The Enterprise Cool URI will save us yet.

Steve Vinoski: The Degenerating ESB Discussion

Dan Hatfield:Honestly, I see the ESB as primarily a political thing. It allows for a greater degree of control on delivered solutions. In large companies, we don’t often do architecture - we do politecture…The politics drive the architecture. Not the way it should be…but that’s the way it is.

Politecture! Ouch. Sad but true. I walk this line on a regular basis. I push the envelope as far as it can go, but politics are ever present.

More Vinoski: Another non-technical way to look at it is from the viewpoint of Clayton Christensen’s classic book, The Innovator’s Dilemma. For quite a few years now, we’ve seen a series of sustaining innovations in the “object/service RPC” line of descent originally popularized by CORBA and COM, both of which built on earlier RPC, distributed object, and TP monitor technologies. RMI, EJB, SOAP, WS-*, and ESB are all offspring in that line, and there are surely more to come. I feel that REST, on the other hand, fits the definition of a disruptive innovation perfectly (and if you’re too lazy to read the book, then please at least follow the link, otherwise you won’t understand this at all). The proponents of the sustaining technologies look at REST and say, “well it can’t solve this and it can’t solve that” and voice numerous other complaints about it, precisely as Christensen predicts they would. But Chistensen also explains why, at the end of the day, any real or perceived technical shortcomings simply don’t matter (and in this case, they’re mostly perceived, not real). HTTP-based REST approaches have a lower barrier to entry and are less complex than anything the sustaining technologies have to offer, and REST is disrupting them, whether all the smart folks pushing ESBs like it or not. It’s not a technical issue, and there’s no amount of technology the non-REST tribe can throw at it to stop it because it’s based on how markets work, not on the technical specifics.

Internet Machine

2007-10-06T10:21:00.000-07:00

Tim Bray: The Intimate Internet

But when the next big thing comes along (and I love this business, because I know it will) you won’t have to rely on the professional noticers to tell you because it’ll touch your life directly.

I love this business too.

This internet-machine is wicked cool.

Pretty Good ESB Spaz Out Underway

2007-10-04T20:47:00.000-07:00

The "always-insightful" Patrick Logan has triggered a nice sized blog spaz out on ESBs it seems.

Good stuff is being discussed. Cool. The ESB Question - Steve Vinoski of former IONA fame.

More insight from Patrick: Properly Striking a Balance Between Shared Agreement and Decentralized Execution

There is plenty more, but you can find the rest pretty easily.

Camel Cast

2007-10-01T23:38:00.000-07:00

James Strachan has a Camel Cast.

Ed (co-worker) and I messed with it a bit a couple weeks ago. It is approachable for newbies in that it is just code. I haven't really gotten that excited about the whole DSL thing, but an Enterprise Integration Patterns DSL sounds nice.

When Ed and I messed with it, it seemed young, but headed in a good direction. There is a lot to be said for simple - just download a jar file or 2 vs. adopting a whole platform. I know for a fact that there are a lot of people out there that would like to stop writing this type of code, but either can't or don't want to adopt a whole platform to do it.

It loks like the Camel docs might be getting rev'd a bit. I particularly liked seeing links to unit tests in the docs.

Good docs rock. Matt Asay says it results in more sales for open products.

These abstractions make it easier, see.

2007-10-01T22:39:00.000-07:00

I came across this on time spent in "modern" Java web dev. Funny stuff.

I rat lol'd at the Spring bit and the Hibernate bit.

Via Buko Obele

Cool Enterprise URIs & Mule

2007-09-30T11:14:00.000-07:00

Patrick Logan expounds on my Enterprise Cool URIs post.

Ironically, Ross Mason commented on Patrick's post & Patrick responded with what we've been discussing lately at work. The ironic part is I spent the better part of yesterday messing with Restlet and Mule (Ross's product).

My point of view on this topic has changed significantly over the years like Patrick's. I have been pro-CORBA, pro-EAI, tolerant of WS-* (couple months where I built a couple of them in 2001), pro-message broker, pro-ESB, pro-JavaSpaces, and most recently pro-please don't hurt the web

I think that if you spend enough time with middle-ware, you eventually conclude that the best approach is to not hurt the web as much as you can and embrace a more-or-less-peer-to-peer model (i.e., how the web works). A year and a half ago, I preferred the message-centric to the service-centric approach. I have concluded that at the time I was misguided in how I thought I was dealing with the fallacies of distributed computing. I don't think there is anything wrong with messaging & there is a place for it for sure - I just think that the dominant integration protocol should be HTTP. I used to see messaging in the center, now I see it on the edges. HTTP should be in the center.

It is probably important to note that I am talking about coarse grained integration ... the integration of large domains (e.g., sales, manufacturing). I've heard this called the "federated ESB" approach although I wouldn't call it that. I've also heard it called the "ABC" model (application, business unit/domain, corporate). I'm focused right now on the communication at the corporate level - across domains rather than within them. I think that HTTP makes sense everywhere, but for lower level integration (A & B) there may be other choices (I still think that conceptually, you can't beat the Tuple Space concept for A - large applications).

Mule is an ESB, but that term is essentially meaningless today. MuleSource also refers to it as an "integration platform" and an "Enterprise Service Network". Mule allows and encourages any-2-any protocol "mediation". This is good. Mule can certainly be used in a way that does not hurt the web.

I have more learning on Mule to do. I was a little overwhelmed in working with it yesterday (lots of jar files, lots of XML config, examples that didn't help with what I wanted to do, docs that didn't answer my questions - typical newbie stuff). I did succeed in getting it configured to do what I wanted it to do after going down a number of rat holes & now have something to build on this week.

Mule may be a good runtime choice, but I agree with Patrick that an ESB isn't your architecture. Your architecture is things like reference data, meta data, the services & events you expose, your URI naming standard, security, and what is on the wire. You want to build these things so they can stand the test of time.

The appealing thing to me about hiding as much of this as possible behind a good URI naming scheme that doesn't change & HTTP is that I know for a fact that the URI & HTTP will be thriving in 10 years. I can't say that about any integration vendor let alone any integration vendor's current product set.

Taming the Traps of Traditional Thinking

2007-09-30T07:44:00.000-07:00

Nice article: Mind of the Innovator: Taming the Traps of Traditional Thinking. It talks about the "seven sins of solutions" & how to tame them.

This is tragic & very true. We all see this:

IDEA loops. IDEA is an acronym for Investigate, Design, Execute, Adjust. it’s a codification of the human learning cycle...the one that starts disappearing around age 5, once we enter the formal school system. that’s when it becomes about the right answer and not the right question.

I have been fortunate in that I have been affected by this less than most. Some of it is personality type, most of it is probably due to really good parents who encouraged me to try things and not be afraid to fail.

We need to stop thinking about innovation as an outcome, and start thinking about innovation as a process. we need to move from innovations to innovation. Because as a practical matter, innovation, problem-solving and learning employ the same iterative process—blending supposition, logic, creativity and reflection

Experiments show that creative revelations come when the mind is engaged in an activity unrelated to the issue being addressed, and that pressure is not conducive to creative thought. recent research demonstrates that the ultimate break—sleep—actually changes our mind’s perspective.

There are pros and cons to being a light sleeper (me). I sleep 6 hours a night unless I'm exhausted. I personally need to be more disciplined in turning things off. Most of the time, software related "work" doesn't seem like work to me because I genuinely enjoy it.

There is this too: Enter the irrational fear of failure. Backing off is counterintuitive. it somehow feels wrong, like preemptive surrender. it’s scary to ease up, because we may lose our steam, or we may abandon hope. we get anxious when the answers aren’t so forthcoming, and we begin to doubt our creativity, abilities and intelligence, fearing that if we take our eye off the problem even for a moment, we may lose the energy we’ve invested.

I definitely have the fear of losing steam - I'm generally not afraid of failure though. I'm a "damn the torpedoes" type. This too has pros and cons.

decisions: "follow the decisions you made" vs. "they were made"

2007-09-29T21:35:00.000-07:00

I saw a link to this on Dan Creswell's del.icio.us:

Train-Wreck Management by Mary Poppendieck.

The article is referenced on InfoQ. The Poppendiecks are my favorite methodologists. I have yet to be disappointed with Lean thinking.

I highlighted some sections that stood out to me:

Exhorting people to "be careful," "try harder," and "work smarter" is not useful if individuals have little effect on results. Rewarding or punishing people for outcomes that are not under their control can only result in discouragement - or in gaming the system. Instead, chronic problems must be fixed by finding their underlying causes and addressing these effectively. As Deming points out, this usually involves changing the system - the way things are done. And according to Deming, it is management's job to change the system.

Deming: All of the empowered, motivated, teamed-up, self-directed, incentivized, accountable, reengineered, and reinvented people you can muster cannot compensate for a dysfunctional system.... A well-run organization with well-functioning systems allows people from top to bottom do work of which they can be proud.

"There is something called standard work, but standards should be changed constantly. Instead, if you think of the standard as the best you can do, it's all over. The standard work is only a baseline for doing further kaizen. It is kai-aku [change for the worse] if things get worse than now, and it is kaizen [change for the better] if things get better than now. Standards are set arbitrarily by humans, so how can they not change?

"When creating Standard Work, it will be difficult to establish a standard if you are trying to achieve 'the best way.' This is a big mistake. Document exactly what you are doing now. If you make it better than it is now, it is kaizen. If not, and you establish the best possible way, the motivation for kaizen will be gone.

"We need to use the words 'you made' as in 'follow the decisions you made.' When we say 'they were made' people feel like it was forced upon them.

Standards are not about how work should be done, but how work is being done. You don't want the standard to be too perfect, because that leaves no incentive for workers to improve their standards.

That is, workers - led by their team leader - do many rapid experiments, find a better way, agree on the improvement, quickly document the new way, and use it. When a standard is improved, the decision for the change must be made by the people doing the work, so they won't feel it is being forced upon them.

When Deming said "change the system", he was talking about changing the complex, interrelated processes used to get work done. Deming believed that changing the system is management's primary job, and in order to do this, managers need competency in four areas: 1. Appreciation for the overall system in which work is done 2. An understanding of variation - and the true relationship between cause and effect 3. Constant pursuit of learning (improvement) through designed experiments 4. An understanding of the psychology of people When all of these areas are balanced and working together, great things can happen.

Ready. Fire. Aim

2007-09-29T08:05:00.000-07:00

David Christiansen sent me a good article by Tom Peters on systems thinking and innovation. I couldn't agree more with it. It is astounding how much time is wasted with excessive planning.

Tom's post reminds me of the book The Myths of Innovation. I read it earlier this summer.

I'm a big believer in failing fast and learning. If you want to innovate, you plan a little bit, create a whole team, give them a wiki, foster a collaborative/transparent culture, set a short deadline, try something, measure it, and do it again until you are done.

I whole heartedly agree with this excerpt: I am an unabashed fan of MIT Media Lab guru Michael Schrage—particularly his book Serious Play. His principal axiom: "You can't be a serious innovator unless you are ready and able to play. 'Serious play' is not an oxymoron; it is the essence of innovation." And, in turn, the heart of his serious play is ... fast prototyping: "Effective prototyping may be the most valuable core competence an innovative organization can hope to have." His intriguing connection, which makes all the sense in the world to me, is that true innovation comes not from the idea per se, though it guides the work, but from the "reaction to the prototype." In fact, in a surprising number of cases (the majority?) the collective responses to a host of fast prototypes reshape the original idea beyond recognition—or lead one down an entirely new path.

A more succinct way to sum up how innovation works is Ready. Fire. Aim. Tom's article says he heard a Cadbury exec call his approach to product development. This phrase is often used as what not to do when it is exactly what you should do.

Update 01-OCT-2007 Tom has some follow up posts that are well worth reading: Systems Thinking II: My Summer Vacation and Systems Thinking III.

Here is a great quote:

"For C-sakes, quit drawing those f-ing maps and run some experiments, quick and dirty, and see if anything you are babbling on about actually works or makes the slightest bit of sense in the real world as we know it. And after you've done your real work, then you are welcome to write your 'complete theory of everything.'" (That was close to the actual script, minus many more f%^*s and about 25 minutes of elaboration.)

Enterprise Cool URIs

2007-09-28T20:32:00.000-07:00

Cool URIs don't change.

We are working on some fairly large scale integration at work.

As you might expect, we are trying to not hurt the web.

As most older "enterprises", we have many many aging systems. But the "domains" or large groupings of our systems are well known & shouldn't change too much over the next 10-20 years.

I'd rather bet on the URI than any ESB vendor.

I started working on a naming scheme for our cool URIs today. I think that if we get that right, we'll be heroes.

With a URI naming scheme that doesn't change you get a very different, very simple view of your systems. Sure, there may be madness today behind those URIs, but over time that madness will hopefully start to go away. But your URIs will stay the same. A simple vaneer on madness that slowly becomes sane. Yay Cool URIs that don't change.

Pair Atom Restlet

2007-09-28T17:22:00.000-07:00

I rewarded myself after a long week of teeing-up the next phase of a big project with some pair-programming with Patrick.

We did some messing around with Restlet & Atom.

It was a pleasant afternoon.

Restlet is a great little API.

Atom is a great little format.

Linux and open source software pay off for PayPal

2007-09-24T23:39:00.000-07:00

Linux and open source software pay off for PayPal: When Scott Thompson left Visa to take the CTO role at PayPal in 2005, the Web company's data centre surprised him. "Wait a minute," he recalls saying, "they run a payment system on Linux?"
"I was pretty familiar with payment systems and global trading systems, but I just scratched my head when I came here," Thompson says. With his history of working on IBM mainframes and large Sun Solaris systems, the PayPal approach to computing seemed alien, especially for a company whose core mission was dealing with money.
PayPal runs thousands of Linux-based, single-rack-unit servers, which host the company's Web-presentation layer, middleware and user interface. Thompson says he quickly saw the economic, operational and development advantages of open source and Linux technology. He now sees no other way to do it.

"When you're buying lots of big iron, as I did in other places I've worked, your upgrade path is $2 million, $3 million at a clip. You just had to buy big chunks of stuff to scale," he says. "Here at PayPal, our upgrade path is 10 $1,000 no-name servers, slapped into the mid-tier of the platform. And we just keep scaling it that way. It's unbelievably cost-effective."

Via Matt Asay.

GOSCON 2007 in Portland

2007-09-23T19:36:00.000-07:00

I got a note from Deb Bryant regarding GOSCON. I went last year and it was great. I'm shocked at how much has happened in a year.

The week after I heard Larry Augustin speak at GOSCON, Compiere was forked. Less than a year later, a VP from Oracle took over Compiere hoping to turn it around.
JanRain was just getting started - I saw Jason McKerr speak at GOSCON. He had recently left OSL which was pretty high flying at the time. I remember him saying, "I'm speaking at an OSS conference, but our products aren't OSS. We did a lot with OSS at OSL." I don't remember much more about what he said. A year later, their CEO has moved on. I have no idea how they are doing besides that. But when that happened, it reminded me just how hard it is to start a company & how hard you have to work to make good software products that the market needs & will pay more $$ than it costs to make. Looking at the JanRain website, I don't know where the $$ will come from. I do think that OpenID is as good an idea as any though for identity. I get really tired of registering and remembering my password. I just don't know how they will monetize it given some of the stuff going on in that community. I hope they do though - clearly some talent there. And I have a natural bias towards all Portland tech companies.
Bradley Wheeler gave the closing keynote at GOSCON - it was incredible.
Stuart Cohen gave a speech as the CEO of OSDL (now The Linux Foundation). He now runs The Collaborative Software Initiave (CSI) - it is conceptually similar to Brad Wheeler's success with universities.
Open Source was a key focus of my daily work (along the lines of Brad Wheeler & CSI)- now it is just one fairly small aspect of it. Open Source has come a long way in a year yet has a ways to go. IMHO it is an unstoppable force. But lest we forget, it only makes up 2% of the software market. My personal opinions around open source have changed a ton in the last year. I remain convinced that it is the #1 disruptive force in the software market, but am a little more realistic about where it is and where it is leading.

Anyway, GOSCON is on again this year (year 3) GOSCON 2007, Oct 15-16 in Portland. Deb says: This year's keynotes include: Andrea DiMaio, Vice President and Distinguished Analyst, Gartner Research; Jim Zemlin, Executive Director, Linux Foundation; Skip McGaughey, Director of Eclipse Ecosystem, Eclipse Foundation.

Roy Fielding Video

2007-09-18T08:27:00.000-07:00

Stefan Tilkov links to Roy Fielding presentation video. It is about 30 minutes long & well worth being late for work over ;)

I wish I could go to ApacheCon to see the latest in person.

Message Queue Panic

2007-09-17T22:23:00.000-07:00

Now Dare Obasanjo is joining the spaz-out re: messaging, push vs. pull and such.

Also, Stefan Tilkov is momentarily a binary messaging bigot.

XMPP is IETF'd. It has been around for years (started in 1998 according to Wikipedia).

AMQP is a young upstart. Perhaps it will break out. I'm just saying it will be hard. It is really hard to get a protocol going. Time will tell I suppose.

At the end of the day, IBM MQ Series/WebSphere MQ still has over 90% of the messaging market last time I heard so in terms of application integration within the "enterprise", I'm putting my money on that continuing for the foreseeable future.

Perhaps overtime, AtomPub, XMPP, and HTTP can put a dent in that.

The whole pub/sub business may be an opening. MQ Series isn't known for that. They have a new 100% Java version that I presume does a better job. XEP 0060, AtomPub, etc. could be alternatives.

In the short term, I think that ActiveMQ has the best chance of taking on IBM MQ Series/WebSphere MQ in terms of the more classical messaging stuff. But then again, I guess that may be ok with IBM as they support Geronimo which includes ActiveMQ.

Seagull Architect

2007-09-17T21:56:00.000-07:00

Speaking of David Christiansen, he recommended a good book recently that I have been reading: Manage It! Your Guide to Modern, Pragmatic Project Management.

He dropped "Seagull Architect" in some meeting we were in a few weeks ago. He got that from the book.

A few excerpts from Johanna Rothman (author):

I've worked on several projects where the architect was like a seagull. He swooped in, dumped a lot of poop in the form of PowerPoint pictures of the architecture, and left as soon as possible. He didn't stick around for the hard part of the project: making the product work in the architecture ore evolving the architecture so that the product could work by the time of release.

. . . But if your architect is overly fond of drawing programs and not fond of writing code and can't really answer the developers' questions about how to make the parts fit into a coherent structure, you don't have a real architect. Eliminate that person from your project . . .

Yep, PowerPoint doesn't compile.

I have been accidentally guilty of this before. My reason was being spread thin, not that I can't sling code. I hated doing that to the team at the time. And it was true, the team really suffered. Ultimately I removed myself from the project and they were successful.

The $100MM Canonical Model

2007-09-17T21:39:00.000-07:00

A co-worker of mine in Indianapolis, IN David Christiansen has a good post with The $100,000,000 Canonical Model.

Lest we forget, there are no sliver bullets.

There are two extremes in my mind when it comes to data formats and integration architecture:

Big Ball of Mud
A canonical model / data exchange architecture will solve everything

The pragmatic approach, IMHO, is to start small and build these assets over time via projects. The wrong approach is to go off for months/years to create it. Clearly, an asset like this needs to be as simple as possible, embrace change, be accessible, and based on Atom. (ok the last bit was bias).

GData AtomPub Podcast

2007-09-17T18:57:00.000-07:00

I just listened to The world of Google data APIs. It is 42 minutes long. I took notes.

Here is the agenda:

What "Google data API" actually means (the parts and pieces)

What Atom, Atom Publishing Protocol, and other tech behind GData are all about

What GData adds to the mix on top of Atom and APP

How Atom compares to RSS

What are ETags? And how can they help me?

Why REST, the style, was chosen for these APIs

Where REST makes sense, and where it doesn't. Resource driven vs. RPC.

What the first GData APIs were

How the killer app of syncing data with Google Calendar

How you actually use the APIs? What do they need to learn? What tools do we give them?

Can you write APIs that implement the same GData APIs?

Notes: Atom is an IETF standard in case you didn't know. RSS isn't.

Google has been working with Atom for over 2 years.

AtomPub provides a basic REST API - Google thought this was a great starting point. Atom leaves query to the student. Google uses URLs to do this. You can have output sorted etc. Atom doesn't say anything about this.

ETags were unfortunately not implemented. They chose a version number in the URL so that multiple people can write to the same entry. Plan to implement ETags in the future.

Are ETags magic? No, a lot simpler than they sound. A little string that tells you the version of the entry. Just a great way of making caching work & making the web more efficient.

Chose Atom because of momentum. And sold on REST. Not SOAP because the web is based on REST. Much easier for devs to learn REST than SOAP. SOAP require tooling. REST is simply manipulating tables of entries.

What are disadvantages of REST? AtomPub is just an implementation of the REST style. Difficult to map certain types of operations to REST - translation API example: send text & send back a list of options & then feedback how to improve the translations. Document centric request / response that was never saved on the server. Just a straight RPC call. REST is about manipulating resources.

How about transactions? Each request is essentially transactional. REST not concerned with multi-resource transactions.

How do devs use? Can use curl if they want to - some do create apps with shell scripts and curl. Have a Java, .NET, PHP, Python, Objective C API. Contributed APIs: Lisp (Patrick!), Ruby, Flash in development. It's just XML over HTTP - it's NOT THAT HARD!!

Authentication - "Client Login" is pretty straight forward auth. URL for uname/passwd where you get a token back that is your identity. Authentication for Web Apps - "Auth Sub" used for "on behalf of" type stuff. You can grant access to other web sites. You control who has access to.

Can I use Google Auth? They are meant to be open standards. Google legal hasn't signed off yet, but licensing will be worked out soon.

What is this "Kinds" business? Entries get passed around a lot - kinds concept is Atom categories to tag each entry with a "kind". Just gives more semantic information to computers (clients).

AtomPub Google Interop Event? 12 devs from different orgs came. Great success. Google's basic AtomPub worked fine. Google custom auth scheme was more problematic. After that everything worked smoothly. A lot of the impls built around AtomPub introspection document - Google doesn't use them much. Will in the future.

Any GData / AtomPub tips? Google has a particular approach to designing APIs. AtomPub good at certain APIs. Things that map well - RPC not one of them. On occassion Atom Entry is just a pointer to the real data (e.g., photos). Prefer to put data in entry as much as possible. There is a lot of art/style/parsimony to it. You can achieve a lot with 1 feed with a lot of query parms. Have to review APIs & ensure that they have good clean concepts: this feed clearly includes these types of elements.

WADL? Haven't found the need for it.

Atom can be tough to get at first, but once you do it is amazingly simple & then applicable to many. Very good programmers who don't understand AtomPub gargen/language can. Concepts are very simple. Feeds, entries, links to other entries. Very simple mental model once you get it. All APIs make the same. Very powerful.

Google working with the IETF on improving Atom/AtomPub. Introduced batch model that increases efficiency a lot. Auth. Teams in Google are very ambitious - hope to make publically available as drafts that can become standards.

Some talk of JAY-SAHN (JSON).

Viking Bus

2007-09-15T13:59:00.000-07:00

Erik Onnen (will answer to "Viking") has a good post on Apache HTTPD & using it as your integration bus.

Day by day I am making a cleaner break with my MOM bigot history.

There is clearly a place for messaging, but I think it's future is on the web. XMPP is of course perfectly positioned for this.

Fixed

2007-09-07T06:53:00.000-07:00

Matt Asay writes about a case against a SI (actually my first employer - it was called Andersen Consulting then):

[T]he U.S. Department of Justice is suing Accenture for allegedly receiving kickback-like payments from technology suppliers it recommended and/or implemented at DOJ. The alleged fraud was a collusion with big-name IT suppliers (e.g., HP, Sun) and smaller vendors (e.g., Vignette) to defraud the Government.

IT land is a very interesting place. It never ceases to amaze me. There is very interesting behavior in this market. There are all sorts of vendors all chasing the same Fortune 500 - 2000. There is tons of money, yet there is none. Everything is easy, yet everything is impossible. To survive/thrive in IT you need to develop many different types of skills. You must, however, maintain your integrity above all else.

Note: Obviously I have no idea if there is any merit to this case. All of the people I worked with and Andersen Consulting had heaps of integrity. I just felt the desire to beat the integrity drum today and saw this and leapt on it.

Viking Async Sighting

2007-09-07T06:28:00.000-07:00

The Viking (aka Erik Onnen) spied AsyncWeb.

From AsyncWeb home page (bold emphasis mine):

The majority of today's popular java HTTP engines are built around the J2EE Servlet Specification - and are inherently synchronous in their operation. The typical order of play is that an individual HTTP connection is mapped to a thread of execution which reads data from the underlying socket, parses the request and forwards it up through the web container to the target Servlet - blocking until a response is completed.

As processing latency (as described above) increases, a real scalability problem is encountered: To increase throughput, it is necessary to increase the number of connection threads. This, however, is not the best way to scale - and depending on the JVM / operating system employed, having a very large number of threads can become a big problem very quickly.

A real problem then needed to be solved: How can we obtain a very high throughput - whilst dealing with high processing latency - without requiring a potentially unbounded number of threads? An initial area of investigation was to provide a NIO transport connector for an existing HTTP engine. However, existing popular HTTP engines built around the J2EE Servlet Specification have a blocking synchronous architecture throughout. Swapping in a NIO transport to such an engine does not solve the problem: We'd still need to scale processing threads in order to push requests up through the container.

What was needed was an http engine which provided non-blocking behaviour throughout - and so work began on writing such an engine.

Looks like AsyncWeb has been heading for Apache MINA for some time. Apache MINA is a network application framework which helps users develop high performance and high scalability network applications easily. It provides an abstract · event-driven · asynchronous API over various transports such as TCP/IP and UDP/IP via Java NIO.

The things I drawr don't come true!?

2007-09-06T20:06:00.000-07:00

Wait, those gigantic diagrams on my wall aren't going to compile with the next version of my bitchn' uber IDE? But the vendors said it was just around the corner!!

"Words and diagrams are all we have to work with. Mr. Klein. We need to live within their limitations. Seek not a single diagram to capture your enterprise architecture. Seek not a single sound bite. It is a hyper-dimensional thing you are grappling with. All you can do is capture fleeting glimpses of it with words and 2 dimensional views. Do not start with the wiring diagram. Mr. Klein. End with that as the least important view of your enterprise architecture. One view amongst many and a not-very-important view at that." Sean McGrath

Via Stefan Tilkov.

Dan Diephouse & Abdera (AtomPub)

2007-09-05T19:03:00.000-07:00

Yay cool beans Dan.

Spring & Abdera

Both please

2007-09-04T22:03:00.000-07:00

Dan Creswell picks up on the INATT business.

The problem is myopic views that focus on either only technology or only business. My point is that the lines are blurring. And the right approach is a balanced view.

If you spent millions and a year on a "build it and they will come" approach to deploying an ESB for your SOA platform, drink 3 shots of "It's Not Not About the Technology KOOL-AID" ASAP.

If you are ignoring the fundamental changes occurring in technology & the relationship society has with technology, drink 3 shots of "It's Not Not About the Technology ALONE KOOL-AID" ASAP.

CouchDB

2007-09-04T06:38:00.000-07:00

I can't remember who's blog pointed me at this, but it is very interesting: CouchDB: Thinking beyond the RDBMS.

There clearly is a need for something like CounchDB. All the impedance of mapping document data to relational is quite tiresome.

I don't think that we'll ever get rid of relational databases all together, but at the very least it would be nice to have something for the cases where it makes no sense to map to a relational database and you want to effectively retain your documents. This could be great just to effectively store service interactions. Is it really just JSON or can I stick XML in there too? JSON is great and all, but there are an awful lot of XML documents out there.

I thought document storage like this was a great idea going back to 2000-2001. I was working at eXcelon (now Sonic Software) at the time. They had a product named XIS that was an XML database built on top of eXcelon Objectstore. It's issue in general at the time was scalability - I am not quite sure what happened to it after Sonic Software (Progress) bought eXcelon.

REST Conversation

2007-09-03T06:48:00.000-07:00

Benjamin Carlyle has a good post on simplifying the introduction to REST here.

I love the question/answer style blog post. I have to do that sometime.