Thoughts From Cory Hamaski..... (Or How Did Chase Misplace $40,000,000,000.00 ???)........

greenspun.com : LUSENET : TimeBomb 2000 (Y2000) : One Thread

[Fair Use: Educational/Research Purposes Only]

November 11, 1999

More near-free software. http://www.oracle.com offers a trial CD of Oracle including the documentation and all the programs and utilities. There's a nominal shipping charge. You promise to stop using the program after 30 days. The program is not time bombed, it will continue working after the 30 day trial.

If you've wanted to get into a high-earning consulting arena, Oracle is a good thing to learn. In addition, if you understand the set-theoretic, relational database model of Oracle and the Oracle query language, you also know Sybase, Ingres, DB2, and other SQL RDBMS.

So here's a plan:

Linux Operating System - Linux Labs - $5.00 CD

Star Suite - http://www.sun.com - $14.95 CD plus shipping.

Oracle RDBMS Trial - http://www.oracle.com - I don't have the exact price.

World Book Encyclopaedia - $30, 4 CD set. (You'll need the W95 that will be on the used computer)

Pentium 120-150 systems with 15" color monitors are $150-$200 from used computer stores (The DeeCee area store is the COMPUTER WAREHOUSE in Falls Church). These will run Linux just fine. Set up a machine, install all the software and you have it. I make it under $300 for a world class computing system with top quality software.

You can use this to learn, Linux (Unix), programming in C/C++, JAVA, Perl, Rexx, SQL and Databases, Office Suites, everything.

None of this is easy but it is cheap.

************

The Chase

I saw the most delightful article in the Wall Street Journal the other day, just a minute, ah, here it is.

I saw it in the print edition and not NEXIS so I'll just paraphrase it. "Heard on the Street" reported that Chase Mahattan's bond balance sheet had a $40,000,000,000.00 discrepancy earlier this year. Youza, that's what Bill Gates would call real money.

The Chase worked on it and got the discrepancy down to $14,000,000,000.00 by September. They spent more months looking through their thousands of bond issues, the WSJ uses the word "painstaking", and have reduced the error to a mere $5,000,000,000.00. The problem started with a software system called "BondMaster" that seems to have gone berzerk. There is no evidence that this is Y2K related nor (or is it or) have the fingerprints of one, Jo Anne Slaven, been found at the scene.

Earlier when I raised the JAE and detailed my concerns. The pollies said that no company could lose track of their balance sheet.

The Pollies need to read WSJ, get out more. Maybe talk to the Chase. They both obviously don't have a big polly brain.

"Heard on the Street" also reported that the Chase found (rather misplaced) another $40,000,000.00 This has something to do with foreign exchange transactions.

Pretty wacky. Computerized systems going bananas at the Chase, they misplace, oh, forty billion dollars, or is it only five billion? Meanwhile, something loony is happening at the foreign exchange desk and the SEC and the Fed are peering in the window.

There's another aspect to this. If the Chase just finished Y2K'ing their systems as our bank pal, JEMie maintains, they've run 100% IVV, *verified* correct, then how the F***ing ding-dong did they lose track of the accounts?

Well, the reasonable answer is, they didn't IVV the wazoo out of their systems. Systems? Ding-dong, it's bozo time at the bank.

Oh the data goes round 'n round round 'n round and it comes out here.

The pollies in denial-ville claim that since the sun came up and my ATM card worked, this is incontestable proof that SAP is the answer, that banks "Get it", and that a college drop-out with a PeeCee can solve the problems because. So there. Oh, and sell real estate too.

No. It doesn't work that way. The problems at, sigh, Hersheys, Bang and Olufsen, Samsonite, Whirlpool, the World Bank, the State of Arizona, and now The Chase are mere warnings. I sense a disturbance in the force.

The main event is yet to come. These early warnings are just that, warnings. Evidence that the world doesn't work the way the pollies think it does. Evidence that banks don't "Get it", that SAP is not the answer, that simple problems can persist for months.

It's *very* amusing that SAP and Banks are both flubbing.

The forty billion dollars that the Chase misplaced is $160 for every person in the U.S. This statement makes as much sense as

"We're 99% done." So? If your heart stops for 15 minutes, 1% of the day, you're still dead.

The problems at the Chase are simply accounting errors caused by software problems. They couldn't resolve it in oh, 2 or 3 hours. The problems persisted for months. The amounts were significant. The Federal Reserve and the SEC are investigating the problems. This was the feature of the Wall Street Journal's "Heard on the Street" column.

I'd be laughing if this weren't so sad. This is playing out worse than I expected. We're in for it.

The management at the Chase is not laughing about this one. There has certainly been screaming and fist pounding. But this is just a warning, an indication that all is not well.

This kind of error could put the most robust and healthy company out of business. For that reason, I urge you to set aside some resources. Organize, draw your strength, get ready to lend a hand. We'll see odd problems in the large systems.

I expect every one of us to be hit by the Y2K failures. These problems will persist and will not be fixed in a few hours, days, weeks, or months. The IT problems we've seen so far have persisted for much longer than that.

Some of the problems have caused significant revenue losses. Y2K will expand the impact to more companies. This is unknown territory.

I'm setting up a couple listservs on www.onelist.com. This is in addition to the public email list, dc-y2k-wrp.

-cory

-- snooze button (alarmclock_2000@yahoo.com), November 16, 1999

Answers

The $40,000,000,000.00 has appeared in my savings acct at Fred's First Bank, Intercourse, PA. Party tonight at Aunt Bea's family restaurant. Free liver n onions to all!

-- (finderskeepars@losers.weepers), November 16, 1999.

Is Cory Polish?

-- (sumomom@weight.watchers), November 16, 1999.

Oh Lord. Here it goes.

-- thanks (for@scaring.crapless), November 16, 1999.

Chase was one of the better IT shops in the late 70's...so sad...maybe Cory is right, embeddeds may be bad, but enterprise systems could be the real killer.



-- K. Stevens (kstevens@ It's ALL going away in January.com), November 16, 1999.


Quick recap: I have seen firsthand the effect of an "ordinary" bank screwup. An uncertainly over a few thousand transaction records from a single day resulted in a month of printing off and checking hard copies by every front line staff member in every branch of the bank I worked for at that time. Part time and reserve staff had to be called up, and it cost them a small fortune, but it never hit the news because no more customers than usual got screwed. Why? Because no error actually occurred, this was just an UNCERTAINTY over the validity of some data.

-- Colin MacDonald (roborogerborg@yahoo.com), November 16, 1999.


Hi, Do you know if that computer system (plus monitor) mentioned can be "had" truly for that low price and then shipped to Southern CT?

-- thomas saul (thomas.saul@yale.edu), November 16, 1999.

Now that the party has wound down here...

I know this much about Chase, although it is second hand. They could use an application of jam as they are toast. Instead of using an outside company to review their Y2K remediation, they used a very young, somewhat inexperienced group of new hires. The results? Bug city. The friend I spoke to was a consultant on site through last month. He advised me that the problems are being discovered during normal operations. No parallel testing. No stepped implementation. Cory knows what the results will be as did my friend. His contract expired, as he had planned, and they begged him to stay on. He wanted no part of it, especially the thought of working in their shop on New Years Eve, a long way from home. Even scarier is that they have basically given up and gone into what he called "bunker mode."
They are resorting to FOF. If they haven't been able to finish, test or claim compliance, why in POLLYMANIA does anyone in God's name think that .gov is going to get finished on time???? OR AT ALL???

Just a cheery thought from a friend of mine who was at COMDEX trading war stories...

John "9.5" Galt

-- John Galt (jgaltfla@hotmail.com), November 16, 1999.

So Chase couldn't resolve it in 2 or 3 days, did they shut down??? Are they out of business??? They are still running, business as usual, and that's WORSE than Cory imagined???? Cory must be a real BITR type of guy.

-- Walt (Walt@lcs.k12.ne.us), November 16, 1999.

Delete please....

As far as I am aware this comes of Cory's PAY section. Subscribers PAY for this server. If you choose to click on that section without paying great, that's your problem. HOWEVER, distributing it like this is entirely unfair.

Just my 2 cents

-- STFrancis (STFrancis@heaven.com), November 16, 1999.


Now let's see if I've got this straight:

Chase made a big error which was not y2k releated. Therefore, y2k will be very bad.

IV&V failed to find the date bug that didn't cause the problem in the first place. Therefore, IV&V is a waste of time.

Despite the size of the error, neither Chase nor the bond market is in any financial trouble. Therefore impacts will be very large.

New SAP implementations have had 3 or 4 celebrity failures, out of 20,000 installations. Therefore SAP is "flubbing" and isn't the answer.

Uh, right. Whatever Cory's on, I want some.

-- Flint (flintc@mindspring.com), November 16, 1999.



I would loved to have seen Flint's reaction if he had been a passenger on the Titanic:

"So we hit an iceberg, big deal. The ship is still moving, no problem."

"OK, so now a couple of bottom compartments have water seeping in. The ship is still moving, no problem."

"OK, so the interior of the ship is waterlogged. The deck where I am sitting is just fine, the ship is still moving, no problem."

"OK, so I've had to re-arrange where I was sitting on the deck, and it may take, oh, 2-3 hours to fix this bump in the sea. No problem."

"[glub-glub-gurgle-gurgle]"

-- King of Spain (madrid@aol.cum), November 16, 1999.

Ah, King....

Once again you have proved that you are really only the King of Flawed Analogies.....

-- Johnny Canuck (j_canuck@hotmail.com), November 16, 1999.


Mr. Canuck,

KOS analogy may not be that bad. Chase like the titanic is to big to stop, it just keeps going and stays up after hitting the iceberg. But they both go down eventually and they take things like passengers or other banks with them when they do. If the $40 billion lost in the electronic void is correct, they really are toast.

-- Metoo (metoo@frelance.com), November 16, 1999.


Hey SNOOZE!

Right at the top of the page you lifted this from, it says this:

==================

WRP Members Only

This is the members only part of the site.

Please do not repost members only information on the Internet

=======================

Nice going asshole. very considerate.

-- plonk! (realaddress@hotmail.com), November 16, 1999.


Metoo -

I didn't realize you had the inside dope about Chase being sure to fail:

>>But they [the Titanic and Chase] both go down eventually...<<

You need to pull your pants up; your assumptions are beginning to show.

Johnny

-- Johnny Canuck (j_canuck@hotmail.com), November 16, 1999.



Purely coincidental timing:

Chase Manhattan's chairman to retire Jan. 1

-- Linda (lwmb@psln.com), November 16, 1999.


Here's Flints comments:

Now let's see if I've got this straight:

Chase made a big error which was not y2k related. Therefore, y2k will be very bad.

(No, being one of the "first and best prepared" Banks, Chase had a huge problem. We don't know if it was Y2K related, but if this is the best of the best, what about all the others? Can you not see a simple relationship between bad output and changing all your code?)

IV&V failed to find the date bug that didn't cause the problem in the first place. Therefore, IV&V is a waste of time.

(Incomplete or incompetent IV&V might be a waste of time. Are you saying all 10,000+ banks have done independent IV&V?) Despite the size of the error, neither Chase nor the bond market is in any financial trouble. Therefore impacts will be very large.

(You don't know what the impacts are for Chase. The Bond Market doesn't get into "financial trouble". The price of Bonds goes up or down. The holders of those bonds might lose money or make money.)

New SAP implementations have had 3 or 4 celebrity failures, out of 20,000 installations. Therefore SAP is "flubbing" and isn't the answer.

(If SAP can't get it right with "Celebrity" clients, how will they address the 19,996 other "nobodys"? Why can't Hershey ship it's Christmas candy? Aren't they a big enough "somebody"?)

Uh, right. Whatever Cory's on, I want some.

(Whatever you're on is obviously irreversible in it's effect on your brain. A picture of a frying egg comes to mind.)

-- Gregg (g.abbott@starting-point.com), November 16, 1999.


Gregg:

Let's see if we can find some common ground here, OK?

(No, being one of the "first and best prepared" Banks, Chase had a huge problem. We don't know if it was Y2K related, but if this is the best of the best, what about all the others? Can you not see a simple relationship between bad output and changing all your code?)

Banks have problems. Remember the S&L crisis? They didn't just get their records scrambled, they actually LOST hundreds of billions. And Chase's problem wouldn't even have broken the surface unless "Heard on the Street" hadn't reported it. Did it even really happen? We don't know, "Heard" is a rumor column. However, I can easily understand how runaway software can screw things up. As an object lesson, it's not bad. As a smoking gun, it's neither a gun nor smoking.

(Incomplete or incompetent IV&V might be a waste of time. Are you saying all 10,000+ banks have done independent IV&V?)

Of course not. I'm saying that looking for date errors that remediation missed or screwed up is NOT relevant to this case. Cory says y2k is not implicated in any way. Knowing Cory's penchant for fuzzy implication, he'd have found *some* way to imply that mucking with Chase's code *might* have caused this problem. He didn't even do that much.

(You don't know what the impacts are for Chase. The Bond Market doesn't get into "financial trouble". The price of Bonds goes up or down. The holders of those bonds might lose money or make money.)

No, all I know is that Chase is still around and by all outward appearances doing fine. And (slowly) solving their problem, tracking down how the accounts got screwed up. Quite possibly nobody actually lost a thing. This happened "earlier this year", whenever that was. According to Cory's sources, 7/8ths of the original error has been rectified.

(If SAP can't get it right with "Celebrity" clients, how will they address the 19,996 other "nobodys"? Why can't Hershey ship it's Christmas candy? Aren't they a big enough "somebody"?)

This is a misreading of what I wrote. MANY SAP clients are celebrity clients. I spoke of celebrity *failures*, not clients. Let me try to phrase it differently here. Of 20,000 implementations (and SAP has a reputation for being a BEAR to implement), 3 or 4 went so badly as to become newsworthy. That's a very small number. This is a .003% rate of celebrity failures. OK, Hershey flubbed it. Nestle's is very happy about this.

Basically, Cory is dazzling us with BS again, lots of subtle implications, false leads, non sequiturs, and synecdoche (in this case, making sweeping generalizations based on a very few *highly* unusual examples). And as usual, he's completely ignoring the fact that errors from screwed-up code are *already* happening. No dominoes.

Gregg, can't you see through all Cory's smoke and mirrors to the core issue -- that NONE of these are date bugs?

-- Flint (flintc@mindspring.com), November 16, 1999.


This affects my judgement more than many of the comments above:
"We concluded that the global community was likely to experience varying degrees of Y2K-related failures in every sector, in every region, and at every economic level. In addition, we asserted that the risk of disruption would likely extend to the international trade arena, where a breakdown in any part of the supply chain would have a serious impact on the U.S. and world economies. We recommended, in light of the potential disruption that Y2K might cause, that the United States take a lead role to encourage and facilitate contingency planning by individual countries, regional partners, and international organizations such as the United Nations. -- Jacqueline Williams-Bridgers, the U.S. State Department's inspector general"
Who are you going to listen to?

-- Reporter (reporter_atlarge@hotmail.com), November 17, 1999.

Moderation questions? read the FAQ