Friday, February 15, 2008
Nominative determinism
The latest to catch my eye was in this quote at El Reg: "The vice president in charge of Red Hat's JBoss middleware Craig Muzilla, said Red Hat is going to undercut giants IBM, Oracle and Microsoft in the saturated and expensive middleware sector."
With a name like Muzilla, I guess working for an OSS company was pretty much inevitable.
DATAllegro, DW appliances and the Oracle lock-in effect
What caught my eye was Andy's observation that "Oracle customers can be a harder sell (than Teradata's, for example) not because of the technology, but because of the weight of stored procedures and triggers ... in Oracle's proprietary extension" (ie PL/SQL).
Just looking at the various technologies people talk about on OraNA blogs, I wonder whether the continuing growth in Java and .NET based applications over the last decade - where many of the business rules are executed outside the database - might hurt Oracle precisely by removing the PL/SQL "stickiness" Andy describes.
Admittedly Apex is a counter example - thoroughly PL/SQL, thoroughly Oracle; no chance of moving Apex clients to another RDBMS yet. And the tendency to provide management interfaces through PL/SQL (or very Oracle-specific SQL extensions) also helps to keep Oracle customers tied down. Which is good for us Oracle died-in-the-wool crowd. Whether the lock-in is good for the customers (or even Oracle) is another question; an existing customer lock-in can feel like a lock-out to new customers.
Any thoughts out there?
Regards Nigel
Tuesday, February 05, 2008
Schema Version Control
- Tom Kyte adds his own twist to this story at Coding Horror.
- JAson Hinds asked about database release management on one of the OTN forums.
- And Syed Jaffar Hussein (the Human Fly) launched a thread on Oracle-L on the subject.
Code control is a walk in the park compared to "schema control" (where the delta , like an ALTER TABLE needs to be developed alongside the new version of the database object creation script).
And "data control" adds even more challenges:
- can you upgrade the data in place? or do you need to create a replacement object?
- Is there enough space?
- Do you need to suspend user access? How long will it take?
- what are the chances of reversing the upgrade if something goes wrong?
- Will you even know if something goes wrong?
- Who decides go forward, go back or muddle through?
A complex application version upgrade should be treated with as much respect as a data migration project. There should be a strategy (sure, it may be a standard strategy). Use standard tools / scripts. The more the process is repeatable - and the more it is repeated - the better. Make sure that all likely error conditions (missing log files, out of DB storage, non-writeable working directories, no listener) are tested for - test the tests!
By the way, the "process" includes all the operating procedures. Make sure that upgrades aren't nannied through by developers. Script the whole thing; have the production DBAs try it out the procedures as early as possible in test to make sure that they understand what's supposed to be happening (and how to know if it worked OK; the best thing is if the script just says:
IT ALL WORKED OK
or
SOMETHING WENT WRONG IN STEP 13 - SEE LOGFILE xxxyyy13
By the time the schema-and-data upgrade is executed for real, nothing - not even the unexpected - should be unexpected.
Monday, February 04, 2008
Patently ridiculous
I followed Vince's link to Groklaw where InterSystems, one of the parties, is asking for help.
One of the parties is DataMirror (now part of IBM, and the purchaser of my former employer Constellar). As tne SQL Group, Constellar had an ETL system that seems to fit the key claims as early as 1994 (to my personal knowledge - it may have been on the market in 1993)
I'm stunned that (apparently) Oracle has already caved, handing over $2 million. One of the reasons I am so surprised is:
- I was actually working for Oracle UK in 1994 - the Constellar Hub (then known as Information Junction) was evaluated for one of my clients (who ended up purchasing ETI instead).
- Oracle UK was involved in quite a few pre-sales to Telecoms, Pharma and others over the period 94-2000.
- Oracle US consultants helped support the product in at least a few US clients.
- Constellar's US office was actually sublet from Oracle Corp, and I know many efforts were made to sell (the product and/or the company) to Oracle in around 1997 timescale.
- Oracle's own WarehouseBuilder must date from around then if not earlier.
Of course, Constellar wasn't alone at the time. We competed over time against ETI, Prism, Vmark/Ardent/Ascential, Informatica, and all the others (as well as the ubiquitous roll-your-own, which is still probably the market leader).
Slipping the litigants $2 million may be cheaper than arguing the case for Oracle - but for the rest of us (especially any cash strapped open source projects) it's not an option. And I'm sure a sharp lawyer at IBM might like a bit of ammunition to see them off. Hey IBM, you're innocent!
So if any ETL pioneer (or their heirs and successors) wants my testimony to fight off JuxtaComm's malicious and meritricious suit - just let me know! I'm available for testing out the trans-atlantic business class beds any time you like...
Update 13 March 2008: more from Vincent here.
Monday, January 21, 2008
RAC Stretch Clusters - a survey
A stretch cluster is one in which nodes are separated - possibly by several miles - mainly as a precaution against a complete data-centre outage. I know of a couple of examples where separations in the region of 20-30 miles (30-50 km) are either in use or planned, and other posters on Oracle-L have mentioned "several" or "a handful" of implementations worldwide.
Obviously, the main performance issue for a stretch cluster is the latency and bandwidth of the interconnect. The bandwidth isn't affected by distance, but the latency certainly is.
I'd be very interested to hear from anyone who has implemented a stretch cluster (in test or production) with as much detail as you are free to pass on, particularly:
- what is the distance between sites
- how many nodes at each site
- is the workload evenly distributed (active/active homogeneous), partitioned (active/active, heterogeneous) or uneven (active/passive)
- some indication of database size and transaction rates (in whatever units are meaningful to you
- Any performance issues?
Either email me (nigel at preferisco dot com) or comment below. I would like to publish the results but let me know if any part(s) of your information is too sensitive to be broadcast, even anonymously.
Thanks in advance...
Friday, January 18, 2008
TiddlyWiki
Today, in one of those serendipitious chances that comes from following a couple of irrelevant but intriguing blog links, I tripped over TiddlyWiki. Rather than giving you a server-side wiki, the whole thing is packaged on the client side. A TiddlyWiki is a single HTML file that contains its own code; which self-edits as you add content. You save the file locally (and can then publish to a website if you like).
As a tool written by web designers it just looks great. You don't just click and get another page; the expanded content (called a Tiddler) zooms out at you. You can choose to expand or collapse whichever tiddlers you like - at the same time. That may be a big advantage over more boring wiki implementations.
For large communities, it's probably all wrong - personal changes in effect fork the entire wiki; but it may be a very effective personal tool - kind of mind mapping on the cheap, but much prettier than editors like treepad (and instantly portable, no software install required - keep it on your USB stick).
Basic usage instructions can be found here - written as a TiddlyWiki so you can try it out. I'm going to see if I can turn a rather unexciting Designer entity report into a thing of beauty...
Wednesday, December 19, 2007
Latest Bloor market research for Data Migration
Generally sensible throughout but - unusually for Philip Howard's research output -it leaves some unanswered questions.
Even as a statistical forecasting effort, it does leave something to be desired. Rather a lot of tables have been included that don't contribute anythingto the sum of human knowledge - in particular a single CAGR (compound annual growth rate) of 10% appears to have been applied willy-nilly to every single region and sector (confirmed by retrofitting the figures -allowing for rounding to the nearest million). What a shame the research doesn't build on either Bloor or external figures for:-
- likely frequency of application replacement (by industry: Banking hasa more rapid cycle than Trading Companies, and perhaps by context/application type)
- industry/regional growth forecasts (eg from OECD)
- No mention of government / public sector projects This is the biggest area for cockups, especially given the fragmented natureof PFI/PPI in the UK, and similar huckster schemes abroad.
However, the headline figures are worth absorbing, to give an order of magnitude for the market, and more importantly an idea of the extent of project failure.
- 84% of data migration projects "fail" (not delivered on time and on budget)
- Cost overruns average 30%; time overruns average 40%
- Data migration is a $5 billion market this year (or maybe twice that if you throw in smaller projects that come in under the radar for this survery) growing to $12 billion by 2012 (sounds like a lot - but London is spending more than that on the 2012 Olympics...)
At a more detailed level, I'm not entirely comfortable with the attempt at separating "data migration" as a market in its own right from related markets (ETL, EAI, etc). I would prefer to see DM as a particular use case for ETL, EAI and DQ tools; many of the same tools will also have applications for data integration, application integration, business intelligence and other use cases - although mileage may vary.
Howard states that DM only aplies to "one off" migrations and is distinct from data integration; well, yes and no. DM may have to take place over months or years; during that period, "source" and "target" systems will have to coexist somehow. That may be achieved through "temporary" data integration/replication, or by partitioning the data and gradually handing over slices from source to target. Given the ever-changing nature of business, it is by no means inconceivable that the process never reaches its originally planned conclusion (the final decommissioning of the source system). Parts of the "source" may turn out to be worth retaining (ie the economic case for replacement is not viable).
Perhaps the biggest concern I have is that there is no mention of DM as a subset of business change. DM scoping is often imposed by (sometimes poorly considered) "business" considerations and decisions. There needs to be a feedback loop from DM processes into the overall project feasibility, scoping, planning, and costing. How many DM budgets are over budget simply because the budget was unrealistic to start with - because project costing was done using some rule of thumb [eg see John Morris's data migration blog] that happened to be inadequate in the circumstances?
Finally, I am amused (and not at all surprised) to find that hand coding is the market leader at 30%, beating ETL into second place with 28%. Given that ETL tools are often given away with applications, databases, and maybe even with leading brands of Cornflakes it is amazing that this situation hasn't much changed in the last 15-20 years since ETL tools first appeared. I always thought that DIY (roll-your-own for some american speakers) was our (Constellar's) biggest competitor back then - and here's yet more anecdotal evidence. Informatica and other tool developers - not to mention application/product architects - need to understand why that should be:
- no tool is a magic bullet: even a market leading ETL tool may make some cases easier, but can make a few (important) cases much harder to manage (if all you have is a hammer, everything looks like a nail)
- tools are expensive to own: they can be expensive to buy, but much more importantly skilled tool users are expensive to train or hire. The migration skill-base is fragmented (what works for PowerCenter is wrong for Oracle Data Integrator; Ascential does things differently from Ab Initio).
- data migration projects are seen as "boring": so may not always attract the youngest, thrustingest, enterpriseyest architects. Like support, a critical area is treated like a leper in some organisations. No wonder the outcomes are sometimes suboptimal. Those cast into the outer darkness of a migration project eventually either transfer internally (and let their skills degrade), or hop off into a consultancy or frelance contract to rent out their newfound skills. Far too rarely does a competence centre emerge.
- application architects (for in-house and packaged apps) often forget the requirement for data migration into their whizzy new systems. Nearly twenty years after it was first released, Oracle Apps still doesn't have a fully supported bulk migration solution for even the most basic data (eg Payables 11.5.10 finally added vendors, sites and contacts but still doesn't support their bank accounts...). What's unglamourous for the end user is equally unglamourous for the product developer, it seems.
Ho ho ho! Happy Christmas everyone
Wednesday, December 05, 2007
I cite, you borrow, he steals...
I find it particularly annoying as Doug (whose blog I read regularly, but whom I have never met in person) seems to be particularly (even pathologically?) careful to acknowledge where he gets his ideas from.
Take care to read the comments as well as the blog itself. This one will run and run...
Happy UKOUG by the way (from darkest Lancashire at the moment)!
Wednesday, November 07, 2007
Tombstone, AZ
A personal favourite was the Boothill Graveyard on the outskirts of Tombstone. It seems to have operated only for a very few years - and most of the occupants apparently died a sudden death - shot or hanged in the main (one is specifically shown as 'died of natural causes').
George Johnson was more unlucky than most. I wonder how they found out their mistake?
Wednesday, October 17, 2007
Got an issue with that?
I could add the example issue tracking system that comes with Oracle Apex - has anyone tried that?
Other wikis with issue management templates / recipes are available - including PmWiki PITS.
Monday, October 15, 2007
Latest Gartner Magic Quadrant for Data Integration
Saturday, October 13, 2007
Is there WebLogic in Oracle's bid for BEA?
So who's left now? Oracle, IBM, BEA; MS IIS for the Netties, and a handful of low/no cost options led by JBoss (aka Red Hat).
Buying BEA presents an interesting marketing challenge in the J2EE app server area - though at least Oracle only has one app server at a time. Thing are much more exciting (ie confused) in the integration / add-ons market; Oracle Fusion is already in a state of upheaval, digesting bits and pieces from Siebel, Sunopsis, Hyperion and others. Throwing in BEA AquaLogic - which itself combines organic development and some BEA acquisitions, I seem to remember - will be "interesting" to say the least.
And don't forget other jewels at BEA. Tuxedo - a real piece of engineering - something Oracle could usefully combine with the best of its own server technology; and JRockit - a high performance JVM.
For those who hate acquisitions, remember: BEA bought in both JRockit (from a startup) and Tuxedo (originally developed at AT&T, bought in 1995)... and come to that, WebLogic itself (in 1998).
Thursday, October 11, 2007
Onwards and upwards
The setting is a rather large and unusual Oracle Apps implementation - client confidential of course. RAC, stretch cluster, massive batch processing, impossible response and resilience targets? Lots for me to (re)learn, and hopefully some useful previous experience to share.
The 250 mile weekly commute is a bit of a pain...
Bottoms up!
Tuesday, September 11, 2007
Cognos partners with Informatica
Interesting that Informatica's front page still highlights the May press release about being granted an injunction against Business Objects, while Cognos includes this release on its front page. A minor difference in marketing communications, or a subtle indication of who is the senior partner?
Analysis from The Street here, and from Intelligent Enterprise here.
Saturday, September 01, 2007
Finished at last (well, almost)
Thursday, July 19, 2007
No more mirror, mirror jokes... IBM snaffles DataMirror
DataMirror never really "got" Constellar. Sure, I think they sweated the customers enough to make a return on their CDN$10million investment (what a steal that was!), but they never made a serious effort (afaik) to bring together the best of the Hub and TS. A shame, because I think it could have been a winner compared to the still rather tired 1990s EAI products like Informatica, DataStage, Ab Initio etc. In fact, TS+Hub could have turned out not unlike the Oracle Data Integrator (but with added "publish / subscribe", and strong IBM and Oracle partnerships).
Now the boot is on the other foot. Will IBM "get" DataMirror and its various product lines? All we can be reasonably sure of is that the Hub is now officially past its sell by date...
Am I glad I left DataMirror in 2001, and so didn't get bought again 6 years later? On balance, I think so. But I know some who have stayed all along - will they stay with IBM too?
Monday, July 09, 2007
Iona FUSEs Celtix and LogicBlaze
CBR points out that Iona seems to be favouring LogicBlaze supported components (Apache projects ActiveMQ, ServiceMix and Camel) over Celtix components (apart from CXF, which has moved over from ObjectWeb to Apache). The LogicBlaze purchase must be working out well, methinks...
Friday, July 06, 2007
Talend - another open source ETL tool
As well as the usual drag-n-drop transformation GUI, this apparently supports business process modelling - which gives Talend a feature that many "real" ETL/EAI tools don't have. There's also support for using a server grid to parallelise processing, and there's an "on demand" SaaS offering.
Version 2.0 is now available, as well as a 2.1 release candidate which is said to add features including:
- further optimizations for performance increase
- support of new databases (including bulk load)
- transaction management (connection sharing, commit and rollback)
- Slowly Changing Dimensions support
- MOM (Message Oriented Middleware) support for real-time integration jobs
- fuzzy logic data matching (using the Levenshtein and metaphone algorithms)
- normalization, denormalization and flow merge
- support of SSH remote connections
- support of PGP file decryption (through GPG binary)
- reinforced support of the XML standard: DTD validation, XSD, XSLT transformation, significant improvements in hierarchical XML file generation, support of the XMLRPC Web Services protocol…
- improvements to the tMap component, to support input filters and new joins types such (the cartesian product, first match, last match…)
Sounds like it might be worth a closer look...
Zoning in to Sonos
We started around Christmas buying 2 Squeezeboxes - because I'm cheap, and because the Sonos system needs at least one box on wired Ethernet - which we didn't have handy in the right rooms. Although the Squeezebox looks good, it didn't really deliver as I had hoped.
The worst things about the Squeezebox:
- it's not very good at coping with poor wireless reception - which probably accounts for most of the other problems
- although you can synchronise two boxes (say one in the kitchen, and one in the living room) they tend to drift apart; not a good sound!
- The slimserver software is incredibly slow at (re)indexing
- the whole solution relies on slimserver software installed (by an importer) on a linux NAS fileserver - that makes getting updates much harder.
- Internet radio was listed, but I could never get it to work
- Documentation and help was rather flaky
- The boxes look great but the UI is crummy - I never managed to explain it to my wife (or even to myself...)
So we cut our losses and invested in two Sonos ZP80s - they're quite small boxes, which feed into your existing amplifiers (you can use ZP100 if you want a built in amp). Rather than recable our house, we got a couple of powernet adapters. Music sits on a (simple) NAS disc - no intelligence required there.
We had some difficulty getting the powernet working - in the end after three hours trying every feasible combination of sockets in our (1st floor) office and (ground floor) living room, the very helpful installer swapped the original Netgear products for Microlink and found a pair of sockets that worked. Fantastic! after that everything was very straightforward. There's a (pricey) i-Pod like controller - or you can use a desktop interface. Once you've sussed the Zones and Music buttons, it's a piece of cake to navigate.
It's using the same music library as Windows Media Player - so I can load music once, and use it either on my headphones from the laptop, or on the Sonos system. Having the same music in two zones is easy - and very reliable. Adding new zones is also very simple (I just know we'll need a third zone in the office...). And internet radio works a treat - though with a slightly robotic quality, from the selection of sites supported (plenty of UK ones - helpful for us as we get almost nothing on DAB or FM).
The Squeezeboxes won't be entirely wasted; they'll go out to Italy this summer, where they should benefit - eventually - from being wired in (assuming the builders have remembered the CAT-5...).
