Got an issue with that?

Here's a little summary of issue management systems I found in the Register for use with (agile) software products. It struck a chord with me - not least because I am right now working with a project team of around 30 trying to manage and organise a range of issues on an ever changing series of spreadsheets, using email as the repository.

I could add the example issue tracking system that comes with Oracle Apex - has anyone tried that?

Other wikis with issue management templates / recipes are available - including PmWiki PITS.

Latest Gartner Magic Quadrant for Data Integration

Informatica has helpfully syndicated the latest data integration magic quadrant. Out goes Ab Initio - apparently due to their perennial problems with secrecy (ie they won't tell anyone anything). Only IBM and Informatica make it to poll position (IBM slightly bolstered by its recent purchase of DataMirror). Hummingbird is the back marker, both for vision and ability to execute. Anyway, read it yourselves; no major surprises in there (and why would there be in such a mature market...).

Is there WebLogic in Oracle's bid for BEA?

Finally, Larry has got out his wallet and made a bid for BEA( which of course they have rejected). This one has been waiting to happen for several years. Oracle's experience with app servers hasn't been entirely happy - but after a slow start they have managed to steamroller their way to a decent market share (often on the back of Oracle Apps); meanwhile most of the (often less well funded) pioneers have fallen by the wayside. Who remembers Persistence? Gemstone? Whatever happened to Sun's variously named products?

So who's left now? Oracle, IBM, BEA; MS IIS for the Netties, and a handful of low/no cost options led by JBoss (aka Red Hat).

Buying BEA presents an interesting marketing challenge in the J2EE app server area - though at least Oracle only has one app server at a time. Thing are much more exciting (ie confused) in the integration / add-ons market; Oracle Fusion is already in a state of upheaval, digesting bits and pieces from Siebel, Sunopsis, Hyperion and others. Throwing in BEA AquaLogic - which itself combines organic development and some BEA acquisitions, I seem to remember - will be "interesting" to say the least.

And don't forget other jewels at BEA. Tuxedo - a real piece of engineering - something Oracle could usefully combine with the best of its own server technology; and JRockit - a high performance JVM.

For those who hate acquisitions, remember: BEA bought in both JRockit (from a startup) and Tuxedo (originally developed at AT&T, bought in 1995)... and come to that, WebLogic itself (in 1998).

Onwards and upwards

I've recently started a new contract which should prove very interesting. Starting off with a data migration study, I'm expecting there will be lots of performance planning and assurance to follow.

The setting is a rather large and unusual Oracle Apps implementation - client confidential of course. RAC, stretch cluster, massive batch processing, impossible response and resilience targets? Lots for me to (re)learn, and hopefully some useful previous experience to share.

The 250 mile weekly commute is a bit of a pain...

Bottoms up!

Cognos partners with Informatica

Cognos and Informatica have announced a strategic relationship; Cognos will sell and support Informatica Data Quality and Data Explorer products, and the two companies will "team to jointly provide customers with data integration capabilities" with a focus on performance management (rather than the wider data integration market?).

Interesting that Informatica's front page still highlights the May press release about being granted an injunction against Business Objects, while Cognos includes this release on its front page. A minor difference in marketing communications, or a subtle indication of who is the senior partner?

Analysis from The Street here, and from Intelligent Enterprise here.

Finished at last (well, almost)

It's finally complete - we're just back from a lovely break in Sant'Angelo at the new house. Not entirely a holiday - there was lots of cleaning to do, and dozens of lights to buy and wire in (more to come later, along with a significant amount of Ikea furniture to be trucked in and assembled - in November we hope). And I spent most mornings working (those pesky deadlines again) proving at least that the EDGE service (no UMTS in the mountains) and a 20 euro for 500Mb web deal from TIM were enough to keep me going.

The garden remains - a couple of 80 year old olives should have gone in this week, and work will start on the rest in the autumn.

Nothing to do now but work as hard as possible to pay the bills!

No more mirror, mirror jokes... IBM snaffles DataMirror

IBM bought my former employers DataMirror earlier this week. Obviously their main interests will be the HA product line (a direct value add for them), and then the Transformation Server (TS) real time data integration components. Others have written here and here, so I won't labour that except to say that as part of the package they have won Constellar Hub (formerly dear old Information Junction - there's a phrase that doesn't escape my lips too often).

DataMirror never really "got" Constellar. Sure, I think they sweated the customers enough to make a return on their CDN$10million investment (what a steal that was!), but they never made a serious effort (afaik) to bring together the best of the Hub and TS. A shame, because I think it could have been a winner compared to the still rather tired 1990s EAI products like Informatica, DataStage, Ab Initio etc. In fact, TS+Hub could have turned out not unlike the Oracle Data Integrator (but with added "publish / subscribe", and strong IBM and Oracle partnerships).

Now the boot is on the other foot. Will IBM "get" DataMirror and its various product lines? All we can be reasonably sure of is that the Hub is now officially past its sell by date...

Am I glad I left DataMirror in 2001, and so didn't get bought again 6 years later? On balance, I think so. But I know some who have stayed all along - will they stay with IBM too?

Iona FUSEs Celtix and LogicBlaze

Iona says it is expanding customers' open SOA choices by bringing together under the FUSE banner products that were formerly part of Celtix ESB, or were acquired with the LogicBlaze purchase this April.

CBR points out that Iona seems to be favouring LogicBlaze supported components (Apache projects ActiveMQ, ServiceMix and Camel) over Celtix components (apart from CXF, which has moved over from ObjectWeb to Apache). The LogicBlaze purchase must be working out well, methinks...

Talend - another open source ETL tool

Bloor's Philip Howard writes at IT-Director about Talend Open Studio - unusual among ETL solutions in that it is a code generator (of Java, SQL, Perl) rather than an "engine" type of product.

As well as the usual drag-n-drop transformation GUI, this apparently supports business process modelling - which gives Talend a feature that many "real" ETL/EAI tools don't have. There's also support for using a server grid to parallelise processing, and there's an "on demand" SaaS offering.

Version 2.0 is now available, as well as a 2.1 release candidate which is said to add features including:

• further optimizations for performance increase
• support of new databases (including bulk load)
• transaction management (connection sharing, commit and rollback)
• Slowly Changing Dimensions support
• MOM (Message Oriented Middleware) support for real-time integration jobs
• fuzzy logic data matching (using the Levenshtein and metaphone algorithms)
• normalization, denormalization and flow merge
• support of SSH remote connections
• support of PGP file decryption (through GPG binary)
• reinforced support of the XML standard: DTD validation, XSD, XSLT transformation, significant improvements in hierarchical XML file generation, support of the XMLRPC Web Services protocol…
• improvements to the tMap component, to support input filters and new joins types such (the cartesian product, first match, last match…)

Sounds like it might be worth a closer look...

Zoning in to Sonos

We've finally bitten the bullet and shelled out for a 2 zone Sonos music system. I see we're not alone; Rob Levy, BEA's CTO, says it is his favourite gadget - and Joel Spolsky has also endorsed it.

We started around Christmas buying 2 Squeezeboxes - because I'm cheap, and because the Sonos system needs at least one box on wired Ethernet - which we didn't have handy in the right rooms. Although the Squeezebox looks good, it didn't really deliver as I had hoped.

The worst things about the Squeezebox:

• it's not very good at coping with poor wireless reception - which probably accounts for most of the other problems


• although you can synchronise two boxes (say one in the kitchen, and one in the living room) they tend to drift apart; not a good sound!


• The slimserver software is incredibly slow at (re)indexing


• the whole solution relies on slimserver software installed (by an importer) on a linux NAS fileserver - that makes getting updates much harder.


• Internet radio was listed, but I could never get it to work


• Documentation and help was rather flaky


• The boxes look great but the UI is crummy - I never managed to explain it to my wife (or even to myself...)

So we cut our losses and invested in two Sonos ZP80s - they're quite small boxes, which feed into your existing amplifiers (you can use ZP100 if you want a built in amp). Rather than recable our house, we got a couple of powernet adapters. Music sits on a (simple) NAS disc - no intelligence required there.

We had some difficulty getting the powernet working - in the end after three hours trying every feasible combination of sockets in our (1st floor) office and (ground floor) living room, the very helpful installer swapped the original Netgear products for Microlink and found a pair of sockets that worked. Fantastic! after that everything was very straightforward. There's a (pricey) i-Pod like controller - or you can use a desktop interface. Once you've sussed the Zones and Music buttons, it's a piece of cake to navigate.

It's using the same music library as Windows Media Player - so I can load music once, and use it either on my headphones from the laptop, or on the Sonos system. Having the same music in two zones is easy - and very reliable. Adding new zones is also very simple (I just know we'll need a third zone in the office...). And internet radio works a treat - though with a slightly robotic quality, from the selection of sites supported (plenty of UK ones - helpful for us as we get almost nothing on DAB or FM).

The Squeezeboxes won't be entirely wasted; they'll go out to Italy this summer, where they should benefit - eventually - from being wired in (assuming the builders have remembered the CAT-5...).

Graphic example

Thanks to Oracle AppsLab for referencing this stunning presentation by Hans Rosling. Not just for the superbly informative animated graphics, but also for the moving and thoughtful content (and the sting in the tail - watch all 19 minutes for the climax).

Could open source BI close out incumbents?

Bloor's Philip Howard writes in The Register that Jaspersoft 2.0 is now a serious threat to mainstream BI (and ETL) vendors; a full enterprise license for $35k will knock spots off most of the competition.

Perhaps that helps to explain why the fizz has gone out of Cognos's recent results.

And it doesn't help that according to a survey sponsored by Sybase and reported by CBR, only 13% of UK BI projects work as expected. So naturally "31% of firms surveyed, particularly small and medium sized businesses, [are] keeping an especially close eye on keeping costs down to a manageable level."

To stakeholder (verb, transitive)

We've all done it at some point in our scribblings; we use horrible new words just because we know that you know what we really mean.

One particularly fashionable word at the moment is stakeholder. It's a useful short-hand for a person or group that has an investment, share, or interest in something, as a business or industry - and which can include employees, customers, the local community - and (if necessary) Uncle Tom Cobbley and all.

Here's a new usage I found in a consultant's report recently:

Customers should ideally not be adversely affected ... and if downstream impacts are identified the customer should be stakeholdered ahead of time and those impacts managed accordingly.

From the context, I think the author means that someone should explain what's happening to the customer, and manage his (understandable) wrath. But rather than being good for the customer, being stakeholdered sounds more like getting out the garlic and the silver stake. Hold your head in shame, anonymous author!

IBM loses 80% of its Informix customer base

Computer Business Review reports that IBM admits to losing 4 out of every 5 customers for the Informix database it bought back in 2001. That's 80,000 gone out of 100,000.

Where did they go - to DB2 (in which case IBM is happy, even if the Informix rump isn't), or to Oracle, MS or the open source competition?

There's a contrary opinion here from ComputerWorld. Informix may have been "IBM's dirty little secret" but CW believes that the recent "Cheetah" release, improved marketing and changes to senior management in IBM will add up to a better future. Oh, and those 20,000 customers turn into 20,000 members of the Informix User Group - so there may be more out there that CBR didn't count.

IBM acquires Telelogic; a Rational move?

Interesting. IBM just announced the acquisition of Telelogic for around $745M. Telelogic is a Swedish company specialising in software configuration management; obviously their products will come under the Rational brand within IBM.

I spent much of the the last year as a Telelogic CM/Synergy user - having previously been more familiar with IBM Rational's own ClearCase product. They each have their strengths and weaknesses; but both suffer most from the naivety of their users and administrators. I've now worked on several sites who have proudly handed over some seriously enterprisey sum of money to the vendor, and then used one or other of these products when CVS or Subversion would have worked just as well (or maybe better). Why buy an all-singing, all-dancing tool, and then tie its hands behind its back?

Anyway, read what the Register says here and here.

The new house is coming on nicely

Here's a view of our new venture Casa dei due Mori - the house of the two mulberries - under construction. We're certainly looking forward to spending lots of time there this summer when it should be finished. It's been a long trek to get this far - nearly four years from the thought to the finished article. I think it will be well worth the wait.

Ciao!

Buffered LOB_APPEND

Jens Ulrik asked to see the code for a buffered LOB_APPEND mentioned in an earlier post LOB_APPEND eats resources....

Here it is as part of a package. Apologies for the rush job at the time; inelegant, but it was worth it...

    gi_chunk_length integer := 0;
    gv_chunk varchar2(32767);

    -- add a character value to the CLOB being built up; catch null values
  -- NOTE - final flush is done elsewhere
  --        and we assume max(length(av_value))  < ki_maxchunksize

    PROCEDURE lob_append (at_doc_clob IN OUT NOCOPY CLOB, av_value IN VARCHAR2)
    IS
     li_value_length integer;
    li_new_chunk_length integer;
    ki_maxchunksize constant integer := 32000;
    BEGIN
        -- prt_common_pkg.print_xml_string(av_value);
        IF av_value IS NOT NULL THEN
        li_value_length := length(av_value);
    li_new_chunk_length := gi_chunk_length + li_value_length;
        
    if li_new_chunk_length >= ki_maxchunksize 
    then
      -- pad out the chunk as much as possible
      gv_chunk := gv_chunk || substr(av_value,1,ki_maxchunksize - gi_chunk_length);
      -- write out
      DBMS_LOB.append (at_doc_clob, gv_chunk);
      -- initialise remainder into next chunk
      gi_chunk_length := li_new_chunk_length - ki_maxchunksize;
      gv_chunk := substr(av_value,li_value_length +1 - gi_chunk_length);
    else
      gv_chunk := gv_chunk || av_value;
      gi_chunk_length := li_new_chunk_length;
    end if;
        ELSE
            prt_common_pkg.LOG ('lob_append null ignored',
                                prt_common_pkg.klog_ret
                               );
        END IF;
    END;

The final flush of the last chunk of a LOB is called like this:

    
    IF gi_chunk_length > 0 THEN
       -- flush out the last chunk of the CLOB
     dbms_lob.append(at_doc_clob, gv_chunk);
     gi_chunk_length := 0;
     gv_chunk := null;
    END IF;

ETI offers data integration projects as a built-to-order service

(Disclosure: I worked for Constellar, a competitor, from 1995 to 2001, and worked with - or perhaps I should say against - ETI*Extract at an Oracle customer site as long ago as 1994).

Bloor's Philip Howard reports that ETI is offering a 'built to order' service for data migration / integration, based on ETI Solution (formerly ETI*Extract). Just enter your requirements into their integration portal, or send them a Word document, and they claim to be able to deliver a generated solution at an average price of $25,000 - and they claim to guarantee a delivery time (usually just 2-4 weeks). For more info, see ETI Built To Order Integration.

In my opinion, ETI's products suffered poor sales in the past from being pricey and (compared to products like Informatica) difficult to use. The template-based generation (by no means unique to ETI - Constellar took a similar approach, as does Oracle Data Integrator) was too hard for most clients to customise. However, the fact that it is a pure generator (there's no runtime engine) helped ETI establish a niche supporting all kinds of legacy/mainframe data sources and targets, and now makes it possible for them to deliver generated code without leaving their IP behind.

Offering BTO seems like a good way to exploit the tool's underlying capabilities, and the skills ETI's own consultants have accumulated, without having to make the tool itself more (dare I say) user friendly.

One question - most integration projects don't know what their requirements really are until they start development and testing. How will this service deal with iterative development? What about late 'clarifications' - "oh, did I mention that the NOTES field got used to distinguish customer types during the early 80s; in the 90s it was used to hold foreign currency information for certain services..."?

So now let's see how this works in real life!

Open Source BI and ETL - picking up pace?

A couple of recent stories to ponder on: SnapLogic raises $2.5M (admittedly from its founder's very own VC boutique) to build its LAMP based data services, while according to CBR, Ingres is preparing a Jaspersoft based BI/ETL appliance.

Meanwhile Pentaho has released its Data Integration 2.5 (formerly known as Kettle) and is showing at ODTUG what it calls "the world's most popular open source BI suite".

How to build a calculator - not!

The wonderful Worse Than Failure (formerly wtf.com) has been running The Olympiad of Misguided Geeks at Worse Than Failure Programming Contest. It's an inspired idea: just how bad can you make a deliverable, while at the same time meeting the requirement.

Alex set contestants the task of building a calculator. Responses have included:

• Driving it all through optical character recognition - my personal favourite so far


• collecting inputs and simply sending them to CALC.EXE to do the work


• one built on an extensible framework that allows you to add new digits. You want to add apples and oranges? well now you can!

All too good to miss - you can find them here, so enjoy!