info 2.0 » enterprise mashups

I love that feeling, when you are talking to someone and the light bulb finally goes on. Even if it’s something you have been talking about and feel like no one gets, and then finally, BAM. You literally can see it happen.

Today was one of those days. We’ve been talking about MashupHub (officially it’s IBM InfoSphere MashupHub now, but we’ll forgo the corporate garb here) for awhile and its capabilities, but to be honest we’ve really been limiting its capabilities to what it will be able to do with Lotus Mashups and a few other products out there - which is awesome stuff, don’t get me wrong. But we’ve got additional plans, right? We do have plans to release a standalone MashupHub (can’t comment on that date yet, sorry). And man, Mike Ferguson just nailed them today. He gets it, and I LOVE IT.

Mike has been writing an ongoing article I’ve been following on mashup technologies, titled, “Web 2.0 and Business Intelligence: How do they fit together?” Now, I’ve never met Mike (I don’t think…) and I opened my reader today to see his article, focusing on mashup products, but specifically calling out MashupHub, and the product formerly known as QEDWiki (these capabilities will be available in Lotus Mashups VERY soon).

Mike writes about MashupHub capabilities here, and says (I have highlighted the important parts):

This server [MashupHub]takes data feeds and mashes them together to provide richer information. Note here that mashups separate the mashing of data from the presentation of data. Therefore, the data can be presented using a variety of mechanisms once it has been mashed…. Various output formats can be rendered by a mashup server (e.g., RSS, ATOM or even XML, which can be passed to browser side JavaScript and XSLT). I see no reason why this output could not be made available to portals, Office applications (e.g., Microsoft Outlook 2007 RSS feeds), wikis and other applications. Therefore, this means that business intelligence can be combined with other information without any programming and made available on-demand to offer up even more valuable information.

There are a few key points here that Mike hits on that we’ve been thinking a lot about:

(1) The separation of the mashing of data and the presentation of the mashed data, which relates to how the information can be output.

(2) The different applications and devices that can consume this mashed data.

(3) Information can be combined together without any programming and made available on-demand to offer up even more valuable information.

(4) Also relevant here is our Open API strategy, which David Boloker and I spoke about in March.

Below is the diagram and an excerpt from Mike’s article, but be sure to read it as well. It’s a terrific read, end to end, but kudos to you Mike. You made my day. /LC

Folks, it is official. They are part of IBM, my division, Information Management. I am looking forward to deep interactions with Don Campbell, their CTO, who will now become part of the technical leadership team for Information Management. Every acquisition brings promise and some angst (technology overlap, new team dynamics etc.) but this one seems to be extremely positive from the get go. More on our dashboarding, OLAP, streaming, text analytics, performance management and the like over the course of next few months from me…

Meanwhile, we also announced a new family of products, branded Infosphere — our warehousing and master data management. Fundamentally, these two, plus our Information Server, now represent the part of our portfolio devoted to comprehensive view of information across the enterprise. While I try to keep marketing to a minimum in my blog, this product family will represent a critical thrust by us.

Also, I was quite excited by a series of announcements in Lotusphere — the annual Lotus conference. In particular, Lotus mashups — which will have close affinity with our Info 2.0 initiatives, and Live Text, which among other things contains my favorite Avatar annotation technology. And yes, some other big events happened this week ($45B big), but let me leave that for a later date.

JackBe’s CTO, John Crupi, wrote up an article for Java Developers’ Journal outlining what was “In” and “Out” in 2008 for knowledge workers. It’s a pretty interesting article.

I borrowed this format to put together a few of my own guesses for what’s in for Web/Enterprise 2.0 for 2008. Here goes:

(1) OUT: IT as the gatekeeper to all information. IN: IT as the enabler of good, secure information to the masses. IT is currently seen as the gatekeeper of all information for enterprise organizations; in 2008 I think we’re going to see more IT organizations putting infrastructure and process into place in order to provide business users (and mashup creators) with this information (or data) that they need to create enterprise mashups. IT will turn from the gatekeeper of all information to the enabler or provider of good, secure information to business users. I’ve been blogging about this for a few months since I saw this as a major forcing function of Web 2.0 at Mashup Camp last spring.

(2) OUT: The business user as an observer. IN: The business user as a active player/participant. As mashups products become easier to use, we’re going to see more business users actively taking part in building mashup apps they might need, whether it be for sales and marketing purposes, inventory processes, etc. The business users are going to be actively participating and developing the apps that they need to be successful.

(3) OUT: Vendor struggles to own the entire Enterprise/Web 2.0 market. IN:  Vendor products and ecosystems working together. Right now I’m not really seeing a lot of vendors partnering up if they are each offering mashup or Web 2.0 platforms to users. What I think we’re going to see more of in 2008 is partnerships starting to evolve in these areas, based on which user types vendors are targeting as well as what features are being offered. I see vendors starting to take steps to do this (including IBM), but the user community will also be a major player here encouraging this to happen at a faster rate. This also could easily play into our current thoughts about Open Innovation.

(4) OUT: Heavy, expensive integration platforms. IN: Lightweight, inexpensive Enterprise/Web 2.0 types of applications. Bottom line: Integration platforms are heavy, expensive, and pretty damn difficult to implement. Enterprise 2.0/Web 2.0 isn’t the answer to all of the problems that a full integration platform can solve, but is sure does alleviate certain pain points in this area.

(5) OUT: “Web” mashups. IN: Data-driven mashups. We keep talking about Web 2.0. But what is really being mashed up is data - and data is where the core value of the mashup really is. Google, for example, isn’t a web/search company, it’s a data company. Companies/users that find ways to utilize this data in ways that cut costs, save time, give them a competitive edge, etc is where we’ll see value in the mashup ecosystem in 2008. This is also where Info 2.0 and IBM Mashup Starter Kit come into play (as you’ll be seeing lots more of in 2008 as well)

/LC

My friend Nils Gilman posted a link to my FB page today from Discover Magazine, titled, “Long Live Closed-Source Software!” In the article, Jaron Lanier compares the the techniques and process of science to that of open source software. Lanier’s argument is that because of the vast amounts of information that groups provide to an open source product, that often schedules and features get pushed to the wayside, and he feels that closed-source software projects tend to produce better results in the long run, as described here:

The open-source software community is simply too turbulent to focus its tests and maintain its criteria over an extended duration, and that is a prerequisite to evolving highly original things. There is only one iPhone, but there are hundreds of Linux releases. A closed-software team is a human construction that can tie down enough variables so that software becomes just a little more like a hardware chip—and note that chips, the most encapsulated objects made by humans, get better and better following an exponential pattern of improvement known as Moore’s law.

I do agree with some of this article and his statement to a certain extent, but what Lanier fails to discuss are the organizations around Open Source software that help to provide the infrastructure and the needs to actually deliver the software, such as the ASF. Eclipse is another example; although I find that Eclipse tends to be more vendor driven than Apache, which is the best example (IMHO) of a true community organization dedicated to delivering quality OSS.

Moreover, as the software products (and projects) become more popular, the demand for new releases and features increases, as does the requirement for better software (developed at a faster rate) from the open source development teams. That said, the projects that tend to be less popular will fall to the wayside - and that’s where Lanier’s point is clear - if a product/project isn’t being accepted and used by the masses, then of course development won’t be as structured or delivered on a timely basis as folks lose interest and developers decide to leave the project.

That’s just my 2 cents. Bottom line, an interesting read. Check it out. /LC

I flew to Raleigh this past week for some heads-down planning on Web 2.0 and Info 2.0. We’ve all been working like crazy lately and it’s not going to let up for awhile. But we have a rockstar team developing Info 2.0 and IBMMSK, and it’s truly a great experience working on this team.

I decided to show you guys that over here at IBM we do, in fact, have fun. And we don’t wear blue suits. And we’re sure as hell not the old-school IBM anymore. So let’s just clear that up right now.

This is the first of several posts I’m going to write up on my travels as I hit the road, and the teams that I visit - - so keep posted to my blog for some great pics and some fun facts about our team.

Here is Vijay Dheap. Vijay is our resident competitive guy (I believe “Strategy Analyst” is the formal title), as well as the proud creator of “Ms. Rita”.* Yep, he develops too, as do most folks on this team. He’s also a huge fan of anything with sugar as the key ingredient (as am I - a man after my own heart). I honestly think he has just two speeds, “on” and “hyper” - so when you meet him, beware.

Mark Heid is our Senior Strategy Consultant, prior to his stint as Director of all things product management over in WAS, and my “partner-in-crime.” When we make action item lists of to-do’s, Mark is jack of all trades over here helping us out with anything and everything Info 2.0. Oh, and plus, I just taught him how to text-message, so make sure not to give him your number - he’ll drive you crazy! Also, do not have dinner with him - he’ll order dessert after dessert and is a terrible influence when it comes to health food (oh wait, maybe that’s me…).

Here’s Vera Plechash and David Sink (or, Mr. Sink as I call him). Vera is our expert on customer and competitor behaviors, and also Raleigh’s local elf - she came in Wednesday bearing wonderful bags of X-Mas goodies for all of us.

David Sink is our resident Grinch (oops, I mean Director of Emerging Technologies for the JStart team). He’s actually hardly a grinch, especially given that he brought us a slew of great pastries to kick off our 8 hour meeting. David’s team is responsible for all efforts around customers - if you’re interested in getting your company “jump started” on IBMMSK, let me know and I’ll hook you up with these experts.

Finally, here is the view from my delayed flight in Raleigh. I had to switch flights from a layover in Chicago to one in Denver, which got me in just about midnight last night. All in all, an awesome trip. Looking forward to more travels.

For v1.1, I am searching for a picture of Rod’s cowboy boots. Rod, you got any of those hanging around?/LC

“*Ms. Rita (lovely Rita, “meter maid” in a too short uniform skirt that will never fly with corporate branding IMHO; sheesh, boys!) which is a configurable “utilization management service” to meter, monitor, and monetize web 2.0 and SOA components, applications or environments.” From: http://susanitsa.wordpress.com/category/sap/

Since we’re in Alpha mode right now, planning for Beta and GA of IBM Mashup Starter Kit (IBMMSK), I’ve been spending a lot of time recently thinking about the pricing that surrounds Web and Enterprise 2.0.

There’s quite a range of products out there, from screenscrapers like Dapper to community tools like Yahoo Pipes and then heavier duty enterprise products like BEA’s AquaLogic Pages, Ensemble, etc. And the price range out there is pretty different too, of course - based on the quality of the tools, what they provide (if they’re enterprise class), the support, etc. These prices range from free to over $30K in some instances.

We’ve been looking at these products for quite some time as we’re moving forward on our IBMMSK front. Of couse, we’re looking at a range of pricing for IBMMSK as well based on what features we’re offering in the product and so on. I run community for our CTO, Anant Jhingran, so my focus has mainly been on the community version we’re looking at, which I can almost with 100% certainty say that we’ll be giving away for free. And of course now, this is downloadable for free. However, much of my time is also being spent looking at the features we have, what type of support is offered, and much more around this for our other versions as well (ie, SMB, Enterprise).

This is where I need your help.

I want to ask you guys out there in the blogosphere what you’d like to see from us. We’ve got some ideas, and we’re definitely closing in on some pricing details shortly (you know I’d give these away if I could, but IBM would probably give me quite the hand-slapping if I did). But our team is committed to giving you guys the best product we can at the best prices (or features in the free version) that we can. Specifically, I’d like to know:

• What features would you like to see in the free version of IBMMSK? For example, connectors to what databases, ability to work with other IBM products (or even other non-IBM products out there in the market), etc?
• If you were going to pay for an Enterprise 2.0 solution, what’s a reasonable amount? What features would you be looking for that are above and beyond what folks are currently offering?
• What type of applications are you looking to build? What can we do to help you guys get this done more successfully?
• Anything else you’d like to see from us.

Like I said, we’ll be making these decisions very soon, and I’d like the community to have an impact on what we decide. I sit on all the groups that are making these choices, so please let me know and I’ll do my best to support what you need (and as our team knows, I can be pretty convincing when I need to be).

Let me know, and if you don’t want to post it in the comment section, shoot me an email at cooneyL@us.ibm.com. /LC

BTW: If your company is looking seriously at IBMMSK, let me know and we’ll hook you up with our JStart (Jump Start) team over here. These guys are a bunch of technical rockstars who can get things up and running with your company.

Check out this video - Anant Jhingran, IM’s CTO (or CTM, which I prefer) talks with James Governor about the shift of the Information Management organization within IBM, Web 2.0, OpenSocial, the relationship between IT and LOB, Open Innovation and much more.

James also gives me a nice plug (thanks!) that refers to a couple quotes and an old blog entry of mine from awhile ago that addresses the cultural shift with IT and LOB, and how IT is no longer the gatekeeper of information, but the enabler of providing this information successfully to their enterprise, the LOB units, and the folks that need it.

I actually had a discussion today with Michael Krigsman of Enterprise Irregular fame where we talked about how this is really necessary across the board in order to make projects successful. Michael actually has a new company out there that is working to help out IT in just this way as well. Very interesting stuff.

I love seeing how we’re all as community and a group of vendors moving towards more openness with our products, our approaches, our visions, and really, our culture. We’re at the tipping point of a major shift here, and I can’t wait to see the outcome. This is one that we all need to shape. /LC

BTW: Anant, if you’re Chief Trouble Maker, can I be your right hand trouble-maker? I am sure both Rod and David would agree with this!!

We were in Stamford CT for the annual SWG Analyst event (see here for some good blogs/videocasts on the event), that Steve Mills hosts. I had the duties to (i) present a chat with the expert session on event processing (ii) schmooze with the analysts (iii) participate in the web 2.0 panels.

On (ii), we had a great version of speed dating — we hung around tables and the analysts came by and peppered us with questions, and on (iii), David Boloker, Carol Jones, Arvind Krishna and I had a great dialog on the barriers and opportunities for web 2.0 in the enterprise, and it was very clear that this is a topic of great interest for the analysts attending the meeting.

Let me spend a few minutes on the chat with the expert sessions. The point we (Bob Madey, VP of Strategy in our WebSphere division, Lisa Amini, Sr. Mgr at IBM Research building System S, and I) were making were simple.
1. The market for event processing has several patterns, some around Business Activity Monitoring, some around IT monitoring, some around program trading etc.
2. These patterns have different architectural requirements — along dimensions of event rate (say 200K/sec for program trading, and 1000/sec for RFID tags in a particular set of distribution centers) etc.; latency requirements (milliseconds to minutes), complexity of event correlation etc.
3. That IBM currently does a lot of event processing around its core capabilities — be it WBM, our Netcool products, or DB2 Data Stream Engine.
4. That we are building to a common architecture
5. That we will continue to add these capabilities around our current customer pain points and leverage partners as appropriate
6. That research is working on the next sets of activities (more massive volume, unstructured information, semantics of events etc.)

We had some very active discussions, and look for more from me on how our architecture and market penetration progresses in the next few months.

So, building upon my last post, let me tell you about the second of the three talks that so impressed me last Saturday.  Andrew Tomkins, whom I had the pleasure of working with while at IBM’s Almaden Research Center, spoke after Alon Halevy.  He gave us some interesting stats on information (text) creation on the web, and concluded quickly that a reasonable fraction of this information can easily be indexed for 12.5K dollars, hence is within the grasps of every small company.  BUT, interestingly, cost is only a small measure of things.  The “knowhow” on indexing is quite rare.  Why?  Because before you can get to the interesting aspects of information retrieval, there is all the crap on the web that has to be filtered out.  And, that is an unbearably high burden on folks trying to do research in this area.  So universities have intellect to contribute, but no clean data to learn from.  In some ways, it is yet another manifestation of the dialog that I talked about, and James Governor talked about.

Andrew suggested that there might a compromise — perhaps query logs can be shipped to the academics so that they can then contribute.  But who wants the AOL fiasco?  Aha, but can’t we obfuscate, encrypt and then ship?  Andrew showed in a paper at WWW2007 how any anonymization technique is highly vulnerable on the web — based on techniques that are well known (such as those described very well in Simon Singh’s “The Code Book,”) or based on people doing vanity queries allowing one to guess certain words.

So the problem of research that is based on real data still remains.  And consequently, the network effects of data will accrue to the few who control it.  Let us see if over time the logjam can be broken.

My third installment, on work by Shiv Vaithyanathan, will come soon.

Normally, I am not a big fan of Saturday conference settings (need I say why? :)  But I had agreed to Howard Ho’s request to speak at the Web Intelligence 07 Conference sometime in April, and when he pinged me last week, I realized it was this Saturday.  Groan.  However, I perked up when I heard who else was speaking.  And the 45 min x 3 that I spent listening to the other speakers was one of the most enjoyable and learning experience I had.  So what got me excited?

The first talk I listened to was Alon Halevy’s talk on Dataspaces.  Dataspaces is basically “pay as you go” integration, as opposed to traditional EII focused on getting it right the first time, resulting in a longer time to value.  I had read his dataspaces work before, so that in itself was not new to me (but I do urge my readers to either read the paper or read his slides).  But in the latter part of his talk, he followed it up with his real-world google experience that blew me away.  There are sooooooo many html tables available on the web, that a lot of web integration on structured data can happen by observing simple join columns — dates, locations, states, countries etc etc…  When you have over 150 million tables available to do experiments, you come up with some pretty smart ways of integration based on “data” (and user query patterns).  He also had a humorous (to us, perhaps not to him) piece on how his fame (and fortune, after all, he is at Google is getting ahead of his achievements — there was a recent article in eweek that correlated his work with the pagerank brouhaha, even though he had nothing to do with it.  And another tidbit on (no guess required here) on how the weekend that google base opened up, it was full of uploaded …

The second presentation was by Andrew Tomkins, ex-IBMer, currently at Yahoo.  And the third one was by Shivakumar Vaithyanathan.  More on these in my next post(s).

All in all, a very enjoyable Saturday afternoon.