Process Patterns

December 28, 2010 – 6:29 pm

Allow me to coin a word:

hack·le /ˈhækəl/, noun - A group of 2 or more people working with little or no coordination on one or  more software projects: Apache is “a patchy” web server built with patches submitted by a large, distributed hackle from around the world.

Yes, there are already enough words in the English language. Yes, this is an extremely cheap and trite way to introduce a post. But tough titty. I need a word for groups working on the same project, and ‘team’ won’t cut it. Why, you probably aren’t asking? Because people working on the same software project aren’t necessarily working as a team.

Teams are especially rare in the open source world where developers are working on their own time and dime, and no one is tracking- much less optimizing- team productivity; everyone’s too busy trying to get their pet project in to the next release. What should we call all of these loosely associated people who happen to be hacking on the same software?

How bout a ‘hackle’?

But most of these hackles- FLOSS hackles included- can organize themselves to significantly improve their overall productivity and the quality of the software they are building. With a little time, effort and possibly expense, these hackles can become a bona fied team.

Don’t think I’m trying to sell you some agile process snake oil, because I’m not. I don’t believe that Scrum, eXtreme Programming, or Unified Process are a prescription for success in any and all software projects out there. In fact, you name a methodology, and I don’t believe in it. But I do believe that there are some valuable practices that pop up over and again in these methodologies. And I do believe that there is some methodology that can help you herd your hackle. More on that below.

Pair programming. Daily status meetings. A ‘product backlog’. Test driven development. Various and sundry UML diagrams. All are common solutions to common problems that hackles are likely to face sooner or later. We know methodologies are nothing more than collections of practices that are supposed to improve hackle productivity. Methodologies that are nothing more than a cocktail of established practices have become common hackle-hold names. These are tried and true methodologies that worked well for the particular hackle/project that some process pusher with a book deal has documented in gruesome detail. But these methodologies won’t necessarily work for the unique challenges you’re likely to face in building your unique software solution.

Don’t worry. You don’t have to start putting together a process from scratch. These cute-name methodologies aren’t all bad. Take a good look at each of them; you’re sure to notice a lot of common solutions to common problems. In fact, you might draw an apt analogy to the programming patterns you (should?) use constantly in your code. Fact is, you wouldn’t be the first to reach this epiphany: http://www.ambysoft.com/processPatternsPage.html.

You’d never adopt a system design whole-hog to sorta/kinda address your users’ needs, yet lots of us try to implement the agile flavor of the day with no thought given to how it fits with your hackle and with whatever it is you’re trying to build. Chances are you’ve already run in to an eager beaver project manager that couldn’t wait to throw everyone in to a bullpen just because XP says so. There must be a better way to get your agile on.

There is. As someone who has implement methodologies in many environments with very different climates, I can tell you that there is a constant temptation to tweek the process you’ve picked whenever one of the prescribed practices doesn’t jive with your hackle. With each iteration- if you’re doing iterations- corners are cut and new practices are added for the sake of practically and productivity. No matter what, you’ll end up with a custom process. Maybe it’s time for us to stop resisting common sense and start designing the best-fit process for our respective hackles from the ground up in terms of smaller building blocks called- say- process patterns. Patterns like iterative development, for example.

So, now that I’ve firmly established ‘hackle’ in the lexicon of software development, let’s use it to define another word that should take a good seat in your vocabulary:

Process Patterns, noun - Common solutions for common organizational problems that are used to build bespoke processes to enable hackles solve unique- and substantially more interesting- problems.

,Wil

By wllm | Posted in Agile, eXtreme Programming, Methodology, TDD, UML, Unified Process | Tagged , , | Comments (3)

Visualize Server Ease

June 7, 2010 – 11:47 am

MakaraI just got back from the International PHP Conference, Spring Edition, and a new company paid my airfare. Makara has asked me to help tell the PHP crowd about their product and establish an OSS community around future projects.

Call me lazy, but I prefer to work on projects that largely sell themselves. Makara is no exception. To boil it down to the bone, Makara is a Linux-based image for the cloud that makes clustering, managing and monitoring your apps on the cloud relatively painless. Fun, even.

What’s particularly interesting to me is that it’s not necessarily a cloud solution; you can run it on bare metal if you so desire. But it really comes in to its own on the cloud. You might say that it is the last mile in your journey to getting your existing apps on the cloud by solving the pesky software problems that come up in cloud-scale deployments.

A public beta will be available soon, but here’s a short teaser list to get you salivating in the meantime:

  • One-click clustering
  • Consolidated log management for clusters
  • Change management backed by SVN
  • Full monitoring with a library of slick-looking graphs to impress people with
  • Images for all major cloud platforms
  • Importable/exportable/resumable application capsules
  • Instant install for any part of the application stack, including PHP 5.3 and MySQL, as well as Java.
  • A bunch of stuff I can’t be bothered to type up right now

Let’s face it: when it comes to the cloud, PHPers are a bit late to the party. Makara will help us catch up.

,Wil

UPDATE: As of November 30, 2010, what started out as ‘WebAppVM’, then was known as ‘Makara’, has now settled on the name ‘Red Hat’. Congrats, guys!

By wllm | Posted in Cloud, Makara, Open Source, Union of RAD | Tagged , , | Comments (0)

Programming the Metacloud

May 18, 2010 – 11:42 pm

A friend of mine who’s been thinking about the cloud for a while now came by last week for dinner. Not only does he grok what I’ve been calling the metacloud, he’s already thinking about (and building) a ‘cloud-native’, object-oriented language to program it. After a couple of beers- both drunk by me- he and I took a flight of fancy listing the features any self-respecting cloud-native language should have. Here’s what we came up with:

  1. Context. An application has to have a way of looking up the resources it needs at runtime. This could be in the form of a library like JNDI for Java, only built in to the language. Moving on. . .
  2. Portable objects. The language shouldn’t care in what memory bank- or in which cloud- an object happens to reside; it should only care about getting a the right reference to the right object and doing the right thing with it. To get full credit for this one, we’d have to figure out some mechanism to leave marshalling/unmarshalling headaches behind us. Misha thinks the answer here is to pass behavior along with data over the wire. My gag reflex kicks in at the thought of all the accompanying security issues, but if security in the cloud were based on trust, open standards, and capabilities combined with disciplined adherence to the principle of least privilege, it just might work.
  3. Orthogonal persistence. OK, I didn’t know what this meant when Misha said it, either. But the concept is simple: no distinction between in-memory storage and persistant storage. The corollary here is that the language would have to have built-in query primitives. And, if LINQ is any indication, that may not be such a bad idea.
  4. Observers. Cloud-native languages won’t have to just support massive parallelism; they will have to warmly embrace it. But few day-to-day problems lend themselves to centrally managed processes, and those problems typically involve heavy computation. Of course heavy computation has its place in the cloud, but most computer resources are actually devoted to distributed data manipulation nowadays. Whether it’s one service in your SOA app talking to another or Facebook talking to Twitter, changes in state have to be propagated to many systems which handle them in different ways. We believe the observer pattern comes in big here. In fact, so big that it may not hurt to have common observer patterns like pub sub and priority queues baked in.
  5. Relationships. OK, here I am pushing my relationships agenda again. To recap, I think relationships like ‘parent’ or ‘customer’ are the most important untrapped- and untapped- construct in OO languages, although I’m still trying to convince my friend of this. It’s not as if the problems that the semantic web solves go away in the metacloud. In fact, with queries as language primitives, relationships may just be the most powerful new innovation in language theory since the turn of the century. C’mon people. Give relationships some love, already.

So, what have we missed here?

,Wil

By wllm | Posted in Cloud, Metacloud | Comments (1)

The Inevitable Metacloud

May 13, 2010 – 1:32 pm

I’ve been thinking a lot about the cloud lately. Let me float a few of my latest ideas here:

The metacloud is inevitable.

The metacloud should- and almost surely will- be open in every way.

No one company/person will build the metacloud.

History will repeat itself.

What is this metacloud I speak of? Well, hang Silicon Valley up on a wall and throw a dart at it. Chances are you’ll hit a company trying to abstract public clouds, private clouds, or both to build a platform for the deluge of cloud-native, yet cloud-agnostic software everyone’s expecting. Enomaly, RightScale, cloud.com. . . the list goes on. And if you want to build your own metacloud, no worries. There’s an API for that. Dasein , libcloud, and even my own Simple Cloud API, for example. While Amazon, VMWare and Microsoft (I’ll spare you the links) feverishly add new features to their closed, proprietary clouds in an effort to differentiate their offerings, some small fries are working on smoothing over the differences.

This is where a good old Linux analogy comes in handy. Early on in the development of the kernel (version 1.2), Linus et. al. decided that portability was important. So they added support for Alpha, SPARC, and MIPS. In doing so, they freed Linux from the clutches of the x86 architecture and turned the hardware in to little more than an abstraction for developers working on top of the Linux platform.

The way I see things, EC2, Windows Azure, Rackspace Cloud Servers, and the rest of the gang are pimping processors. They add one feature after the next, while a bunch of startups are working on platforms that solidly hit the 80% of features and leave the developer/user blissfully unaware of where their virtual computers are actually running.

So you see where I’m going with this. The metacloud will be like Linux, abstracting out the differences in cloud computing architectures. And it’s inevitable. It’s just too useful not to be done.

But I have bad news for any metacloud developer who aspires to write the next Linux: Linux wasn’t built in a day, and it sure as hell wasn’t built by a single company. Its popularity came at the cost of almost total freedom. So any company could use it in any way, and many companies decided to back it by adding drivers and modules for their products that wouldn’t be there otherwise.

I believe the metacloud will be developed in the same way. Since API convergence is convenience in the cloud, I also believe that there will eventually be one API to rule them all (Yes! I finally found a place to bust out this reference where it actually makes sense!).

So, there’s a lesson to be learnt from history here. The vendors that made big bucks off of Linux did it by getting in on the community action and putting it to good use in their products, not by building it themselves.

That said, someone’s got to get the ball rolling.

,Wil

By wllm | Posted in Cloud, Metacloud | Tagged , , | Comments (4)

Hello Lithium!

March 26, 2010 – 10:10 am

That’s right, I’m working a new framework. In fact, I’m working with the boys from CakePHP on the codebase they originally intended to christen CakePHP 3. Ultimately, they felt they had to take the codebase elsewhere because they found they couldn’t explore their new ideas under the umbrella of the Cake project. Tell me about it.

There’s even a little story behind our collaboration. I first conversed with Nate Abele and Garrett Woodsworth in the flesh (I’d had a few back and forths with Nate online, but never in person) at a diner around the corner from ZendCon after the final keynote. In fact, Nate was on the keynote panel, and I had just stood up and questioned his integrity in front of hundreds of PHP’s brightest stars. The issue at hand was whether it is ethical for project leads to influence polls about their own projects by “bringing out the vote”. Nate said ‘yes’. I was obviously in the ‘no’ camp. This was the first time I spoke to him in person. Amazingly, it wasn’t the last.

Back to the diner. It was a big congregation with 20-30 of the highest profile PHPers out there. We’d all decided to go to Joe’s as a kind of impromptu farewell lunch. Basically, if someone decided to bomb that diner around noon on Thursday, 10/22/2009, PHP would have been totally screwed.

Everyone was very sleep deprived and one-by-one people got up to head to wherever they call home. But Nate, Garrett, and I somehow felt that we had something to talk about. Indeed we did. They told me about this new framework they would be announcing later that day. A framework called Lithium.

Turns out, they were working on a lot of the same ideas that I had been entertaining. In fact, they had already founded a group of PHPers with a mission to make development fast and fun: the Union of RAD. I looked in to both the project and the collective over the next few weeks. I still have a hard time believing that the stars aligned so perfectly. Lithium = Cake experience + Zend experience + a entirely new approach to PHP development? Count me in.

Although Nate and I found a lot of common ground in our attitudes toward writing and documenting code, we never came to see eye-to-eye on project polls. So, we decided to create a poll to see what you think. The results are in:

I hope the rest will be history.

,Wil

By wllm | Posted in Lithium, Union of RAD | Comments (3)

Goodbye Zend

March 25, 2010 – 1:49 pm

You read it right. With the turn of the year, I turned a page in my career. I really wish that didn’t rhyme.
It’s been almost 3 years since I joined Zend as the project lead for Zend Framework. I was introduced to the ZF community as an outsider; I had used Java throughout my career, and what little PHP I had used was framework-free. But since then I’ve made many friends in the ZF community and the larger PHP community. And, even though I have a shiny CS degree (which is rare in the PHP community) I’ve learned more in working with these people on ZF than I have in any other projects to date.
In particular, my friends in the PHP and ZF communities have taught me to question conventional wisdom and consider all options- including options that are somewhat less than elegant. They have shown me that the hallmark of great technologies isn’t how much you can twist your code into a logical Gordian knot, but how they fair when the rubber hits the road. Sounds a lot like hacking? Maybe. But Alexander the Great got the job done on schedule and within budget with one huge hack of his sword. Who are we to judge an app that works well and serves the needs of its users?
Moreover, my ZF and PHP friends have demonstrated how great software can be written by a large and diverse community of developers with a healthy dash of openness. I won’t kid you- it definitely felt like herding cats at times. But I found that we worked best when we embraced the chaos and let transparency work its magic.
The ZF team and community accomplished a lot under my leadership that I’m extremely proud of. The following achievements are just a sampling of what we’ve done in the past few years:

  • 4 big feature releases: 1.5, 1.6, 1.7, and 1.8
  • Zend Framework Certification
  • Implemented an agile process for the Zend Team that resulted in bi-weekly bug-fix releases
  • Explosive growth in adoption (12x by some metrics)
  • Full overhaul of the ZF site
  • Warm and very productive partnerships with Adobe, the Dojo Foundation, Microsoft, IBM, Appcelerator and others

I’d also like to personally thank:

Now on to other adventures. . .

By wllm | Posted in Lithium, Zend Framework | Comments (2)

ZendCon gets its Cloud On

October 18, 2009 – 11:56 pm

I’m a busy guy this week. I’m talking about PHP and Platform Independence in the Cloud at ZendCon, as well as sitting on the keynote panel about Developing on the Cloud. The Simple Cloud API site is due for an update, too.

But I’m not the only one busting ass to bring the cloud to PHP. If you’re at ZendCon, check out these sessions to scratch your cloud computing itch:

Finally, make sure you say Hi. I’m never too busy to meet new peeps who will listen to me going off about about cloud stuff. :)

,Wil

PS: Testing update for @HighTechDad. If you see this, it worked!

By wllm | Posted in Zend | Tagged , | Comments (1)

The Simple Cloud API has landed!

September 26, 2009 – 9:47 am

OK, a lot of you guys have noticed that I’ve been uncharacteristically silent for the past month or so. Now I can finally start talking about it- the Simple Cloud API has launched at http://www.simplecloud.org.

This API gives PHP developers one interface to code to for what we call ‘cloud application services’. These are runtime application services like nosql databases, storage, and simple queues. These services make it easier to write ‘cloud native’ applications that can take full advantage of the cloud’s promises of scalability and high-availability. I’ve found myself saying this a lot lately: “What’s the point of an infinitely scaling compute cloud when your applications can’t scale with it?”

We expect to be adding more of these services to the API as vendors roll them out. Possible services include email, session management, single sign on, etc. I can go on and on, but why don’t we let the vendors figure out what their customers need and vote with adoption? In any case, once such a service hits critical mass, the Simple Cloud API will have you covered.

The Simple Cloud API will be delivered as a new Zend Framework component called Zend_Cloud. The code will be available *very* soon.

So, why didn’t we do elastic compute cloud services in this first version of the API? That deserves a blog post of its own. :)

,Wil

By wllm | Posted in Simple Cloud API | Tagged , , , , | Comments (0)

Zend Framework Knocks Out Another Amazon Service

August 10, 2009 – 11:11 pm

With the new Zend_Service_Amazon_SQS component in Zend Framework 1.9, ZF is quickly becoming the tool of choice for PHP developers building applications for the Amazon flavor of the cloud. This brings us up to full EC2, S3, and now SQS support, and I’m working with ZF’s best to fill out this list with SimpleDB in 1.10.

While elastic compute clouds have gotten most of the attention in the deafening cloud buzz of late, runtime services such as hashtable databases, robust storage, and simple queues are equally important IMO. After all, what good is an infinitely scalable compute cloud when your applications can’t scale with it?

You’ll hear more preaching from me on this subject in the coming months. For now, you can start getting the runtime cloud religion by checking out my cloud service primer.

Later.

,Wil

By wllm | Posted in Zend Framework | Comments (0)

Another day, another blog. . .

August 10, 2009 – 10:43 pm

Howdy all!

Lately I’ve found that I have lots to say about PHP and cloud technologies and not enough places to say it. Hence, this blog.

Please check back early and often for news about stuff I can’t tell you about at the moment.

I can’t guarantee it will be entertaining. I can’t guarantee it will be enlightning. Hell, I can’t even guarantee that it will be informative. I’m just hoping it won’t be incriminating.

Now sit back, lower your expectations, and try to enjoy yourself.

,Wil

By wllm | Posted in Uncategorized | Tagged , , , | Comments (1)
Follow

Get every new post delivered to your Inbox.