XML-DB Mapping Project Is Done

Exception:

1. the address tags verification.
2. the address parsing and elements verification.
   
3. incorporate results to dashboard.

I will visit these left-overs once the higher up approves the time for testing the other originators.

Edwards Ends Second Run for White House

According to NPR: Edwards ends his campaign for the White House.

So depressing... the Dem's horse race is not going to be exciting anymore.

What Did I Do Today?

Everything went so fast, I didn't have time to remember them.

Back To SoCal

Arriving at ONT at 10AM, show up at the office around 11:30AM.

While finishing the NGS Policy XML - database verification, I uncorked nasty application behavior. Consider:

<party>
  <name>Acme</name>
  <address>Address1</address>
  <address>Address2</address>
</party>

Normally this XML is transformed into:

Party table
PartyID	Name
1	Acme

Address table
AddressID	PartyID	Address
1	1	Address1
2	1	Address2

But nooooooo, the wise developer did this instead:

Party table
PartyID	Name	Addr1	Addr2
1	Acme	Address1	Address2

Now I have to think about catching the wrench before it hits the wheels.

Anyway, mum bought a big painting on Sunday. Since it didn't fit into Lisa's car, I was volunteered to transport the items.

I managed to get some rest around 10PM.

Long day - it is.

Flying To San Jose

Thank god the flight is not delayed, make it to SJ only half an hour late.

I Am Swimming Through Haze

It was cold this morning .... I had severe headache from waiting in the platform for only 5 minutes, and napped through the trip.

Work was all about broken logic and fighting viral influences.

Laziness Has It's Virtue

Yesterday, I blogged OCTA's scheduling mishap and boing2 just posted this: Mathematics of waiting for the bus.

The Zorg ZF1 Commercial

Thanks to io9:

Why Convert XML Data To Hash Structure Using XML::SAX?

Instead of using XML::Simple? The structure, generated by XML::Simple, is not consistent since it recognizes an empty tag as empty hash structure, instead of null string.

1. Consider this XML data:
   

   <Order>
     <Number>01234</Number>
     <Status />
   </Order>

2. Using XML::Simple produces:

   $VAR1 = {
     'Order' => {
       'Status' => {},
       'Number' => '01234'
     }
   };

3. Utilizing XML::SAX, we could insert a null string instead of an empty hash structure:

   $VAR1 = {
     'Order' => {
       'Status' => '',
       'Number' => '01234'
     }
   };

Remember: it is simpler to assume that all end of hash structure branch is represented by ref($hash) eq "".

How To Slow Cook A Piglet

This is how the pieces are glued together:

1. Consider the original XML data:
   

   <Order>
     <Number>01234</Number>
     <Status>OPEN</Status>
   </Order>

2. The XML data is translated into Perl data structure, utilizing XML::SAX(why?):

   $VAR1 = {
     'Order' => {
       'Status' => 'OPEN',
       'Number' => '01234'
     }
   };

   The unordered hash index will not mangle the report anymore since:


3. Along the way (of parsing XML data using XML::SAX), a report template is created with references at the right places.
   
   

   	Status	Notes
   <Order>
   <Number>01234</Number>	${$REPORT[0]}{'RESULT'}	${$REPORT[0]}{'NOTES'}
   <Status>OPEN</Status>	${$REPORT[1]}{'RESULT'}	${$REPORT[1]}{'NOTES'}
   </Order>



4. To the bridge the result of comparison and the report template, another data structure is created (will be called: bridge) and it has an exact hash structure as the XML data, but it is holding references to the report template.

   $BRIDGE1 = {
     'Order' => {
       'Status' => '$REPORT[1]',
       'Number' => '$REPORT[0]'
     }
   };

5. Consider the check data for the XML data above:
   

   <Order>
     <Number><SQL>'SELECT Status FROM Order WHERE ORDER_ID = <value>'</SQL></Number>
     <Status><SQL>'SELECT OrdNum FROM Order WHERE ORDER_ID = <value>'</SQL></Status>
   </Order>

   Readability is the main reason the check/validation is written in XML format.


6. Converted to Perl data structure using XML::Simple

   $CHECK1 = {
     'Order' => {
       'Status' => { SQL => 'SELECT Status FROM Order WHERE ORDER_ID = <value>'},
       'Number' => { SQL => 'SELECT OrdNum FROM Order WHERE ORDER_ID = <value>'},
     }
   };

7. Iterating through the XML hash structure, while maintaining same path of traversal with the bridge and check hash structure:
   

   $XML{'Order'} 
     -> $BRIDGE1{'Order'} 
       -> $CHECK1{'Order'}
   
   $XML{'Order'}{'Status'} 
     -> $BRIDGE1{'Order'}{'Status'} 
       -> $CHECK1{'Order'}{'Status'}

8. Find the end of XML hash structure branch to extract the value.
   

   if (ref($XML{'Order'}{'Status'}) eq "") {
     $XMLValue = $XML{'Order'}{'Status'};
   }

9. Get the SQL syntax from check hash and get the content from database.
   

   if (exists $CHECK1{'Order'}{'Status'}{'SQL'}) {
     $SQL = $CHECK1{'Order'}{'Status'}{'SQL'};
   }
   else {
     # bitch later: no SQL defined.
   }
   $DBVALUE = getValue($SQL);

10. Compare the content from XML hash structure and database and write the result out to $REPORT[$index], which provided by bridge hash.
    

    if ($XMLValue eq $DBVALUE) {
      eval "\${".$BRIDGE1{'Order'}{'Status'}."}{'RESULT'} = 'PASSED';";
      eval "\${".$BRIDGE1{'Order'}{'Status'}."}{'NOTES'}  = '$NOTE';";
    }
    else {
      eval "\${".$BRIDGE1{'Order'}{'Status'}."}{'RESULT'} = 'FAILED';";
      eval "\${".$BRIDGE1{'Order'}{'Status'}."}{'NOTES'}  = '$NOTE';";
    }

11. After traversing all of the branches of XML hash structure, open the report template and evaluate template, producing:
    
    

    	Status	Notes
    <Order>
    <Number>01234</Number>	PASSED
    <Status>OPEN</Status>	PASSED
    </Order>

    
    

Scheduling Mishap

Oh boy...

Stepping out of Metrolink #682 only to find the buses were gone. The train was only 5 minutes late! I was wondering, these buses were supposed to be there for us (the Metrolink riders), if they decided to depart without us, we needed to wait for another 1 hour for the next one.

What were those people thinking?

Luckily, the next OCTA 471 SB arrived early and the driver decided to send us to work immediately, rather than waiting for another 40 minutes. Others were not so lucky: one of our developer decided to walk and the bus passed him. Tried to help, but the bus couldn't make safe stop for boarding. The others (470) were still waiting when we departed.

Need To Glue All of The Pieces Together

These items are created:

1. Data - contentRef.conf (PDS)
2. Report.Template.html (HTML)
3. Bridge - reportRef.conf (PDS)
4. Report.html (HTML)
   

Need to glue them all together to generate report, may take several days to work through the exceptions.

Bernancke Refuses To Say R....

From The Economist:

Weekend In Simi Valley

I have to go and spend the weekend in Simi Valley after AWOL for 2 weeks.

Breakfast: hash, sausage, eggs and other non-photogenic leftovers.

As usual, dinner at Little Tokyo, and it was cold!!

And this is odd: several Target stores don't have thermal undies.

Work Work Work

Alright, someone is replacing a network card on production system. Conducting test to see if there is any interruption of service.

Testing is done at 11:30 and the systems is good.

Convert XML to Perl Hash Using XML::SAX

Finally, the XML is converted to Perl data structure through XML::SAX.

The content of contentRef.conf is equal to XML::Simple.

XP SP2 Does Not Have WPA2 Driver

Have to hunt the fix (KB893357) using Google.

Diagram

Data(XML)                                  
|
|XML::SAX                           ---
|                                   |s|
|-> Data - contentRef.conf (PDS) -->|c|
|-> Report.Template.html (HTML) --->|r|
|-> Bridge - reportRef.conf (PDS) ->|i|-> Report.html (HTML)
                                    |p|
|-> Validation (PDS) -------------->|t|
|                                   ---
|XML::Simple                         |
|                                    |
Validation - CHECK.XML (XML)      Database

XML::SAX for Comparison Report

Toying with XML::SAX again since the comparison report has to reflect the original XML.

[Insert sample here]

Currently I am writing a script to mimic XML::Tidy.

Dogs Eat Dogs

Sun to buy MySQL AB for USD 1B
Oracle to buy BEA for USD 8.5B

See The Concept Takes Flight!

Yes, the concept works.

But it flops badly when the comparison report is generated. The regenerated XML is different from the original one, since standard Perl hash doesn't have the index ordered by insertion.

Thinking...

Unordered Hash Is Killing XML::Simple

One of my big irks about Perl: the index of the hash is not sorted automatically by the order of insertion. Yes, Tie::DxHash should do the trick but XML::Simple doesn't that module.

Maybe it is time to start looking into XML::Simple and see if I could replace the conventional hash with DxHash.

US Unemployment Rate Jumped To 5%

Whoa! According to The Economist:

The unemployment rate in America jumped from 4.7% to 5% in December.

American employers, excluding farms, added just 18,000 workers to their payrolls in December, the smallest increase since August 2003. Government hiring more than accounted for the rise: private-sector payrolls fell by 13,000.

OCTA #471 NB

3:53
4:03
4:38 <- the sweet spot
4:53
5:05
6:15

Public Transportation

Today I managed to haul my ass up and took the train.

Left the Sierra Madre Villa Station at 5:45, arrived at LA Union Station around 6:15.
Riding 682 it left for Tustin at 6:45, and arrived at 7:42.

The damnest happened when I was to take the local bus to work. First, I asked a bus driver, which bus should I took to take me to Dyer and Red Hill, the driver shrugged. Consulting the OCTA bus map, it informed me that I need to take bus #471, the same bus which driver shrugged at me. Damn.

Adding insult to the injury she didn't tell me to keep my money, instead she let me to put in the fare, which is free for people who take Metrolink. Double damn.

In anger, over the driver's blatant show of stupidity, I dropped a 1.25$ worth of Metro token and it was accepted as 0.75$, I forgot that I was taking OCTA. Triple damn!

[ .. work .. ]

[ lunch with Anup, ruined my diet by hogging down BJ's Shrimp Scampi ]

[ .. work a little .. ]

[ .. cake cutting for Rose ..]

[ .. work some more .. ]

Everything was going well on the way home, this time I had a guide.

Jen told me that it is generally better to take the OCTA bus #471 at 16:38 than 16:53, since the later one could be running late and there are no more Metrolink train after #689 at 17:16.

iPod Hangs

WTH? I had a Muvo FM TX 1GB and it never - ever - hangs.

Remedy

Comparing Data Between XML And Database

There is a need to instantly verify the content in XML with the content that's being pushed the application to the database.

The project started this Tuesday.

I started with converting the structure of the XML data into Perl hash using XML::Simple. Also, the data structure could serve as the validation data.

Consider this XML data:

<Order>
  <Number>01234</Number>
  <Status>OPEN</Status>
</Order>

Converting the XML to Perl hash using XML::Simple (with KeepRoot => 1) yields:

$VAR1 = {
  'Order' => {
    'Status' => 'OPEN',
    'Number' => '01234'
  }
};

The hash structure could be manipulated as validation data:

$VAR1 = {
  'Order' => {
    'Status' => 'SELECT Status FROM Order WHERE ORDER_ID = <value>',
    'Number' => 'SELECT OrdNum FROM Order WHERE ORDER_ID = <value>'
  }
};

Iterating through the data structure to extract the data and the SQL statement to extract content from database, it is feasible to validate the entire XML structure provided a validation data with similar structure using an algorithm.

NetBIOS Over TCP/IP

We just endured agony from dealing with the team on the other side of the pond. Problem is relatively simple: the India team need to be able to access a shared folder serviced by a Bufallo NAS system, but we want to make sure that the port are opened on the corporate firewall.

First problem: someone changed the corporate WINS, and somehow the clients are not aware of this (even if it refreshes IP address). So \\MachineName is out, must use \\IPAddress instead, lucky that I setup the machine with static IP address.

Second problem: Since the Bufallo NAS is running Linux OS, the machine is offering shared folder service by talking SMB (old version). This poses a grave problem, it doesn't response on port 445, it only replies when someone talk to port 135-139. First the firewall administrator only open port 445 for TCP/UDP, this problem is solved quickly after mentioning Linux box and SMB.

Third and bigger problem: we didn't know the configuration of the PC on the other side. Normally the default advanced settings of TCP/IP looks like this:

But, the PC is setup with a disabled NetBIOS over TCP/IP. Dan found it through network sniffing and this link.

AAA Seafood Restaurant

220 W. Garvey Ave.
Monterey Park, CA 91754
Tel: (626) 571-8898

Daily: 11AM-10PM

It claims "Most Authentic Chinese Cuisine in Town". Well, it probably rings true for Teo Chew person like me. My family always frequent this restaurant since it serves Teo Chew heritage food (deep fried shrimp cake, fried noodle and sticky yam).

Unlike ordinary shrimp cakes, these ones are fully covered in thin gelatin and deep fried, hence it is chewy and moist when bitten into. Pay attention to the temperature before you eat it and you wouldn't burn your tongue.

The deep fried shrimp cakes need to be ordered a day in advance since the cook likes to keep it fresh. It pays to call since the restaurant frequently hosts wedding parties.

Bad Morning

I woke up with a severe case of achy joints... apparently I caught bugs while I travelling to Union Station.

My plan to take public transportation to work had to be postponed.

Union Station

Today, I decided to take Metrolink Gold Line to Union Station and see if I could get myself a ticket for tomorrow.

I was in limbo in when I arrived in Union Station, since all information booths were closed except for Amtrak's. There were several Metrolink vending machines and I tried to to buy a ticket from it, but it was spitting out irrelevant train information (apparently it was based on the the day's schedule instead of general information) causing me not to buy the ticket.

But the trip is not a total lost, at least I get a tour of the station and it was useful the next day.

Since I didn't get the information that I wanted, I decided to call Jen for advice on: which types of ticket should I buy, why the monthly ticket is not available, and how to validate the ticket.

On my way home: bought groceries, eat and sleep since I got a splitting headache.

Sin Cia Bazaar 2008 Get A Green Light

Magellan CrossOverGPS (Preliminary)

I couldn't remember exactly when I decided to settle with Magellan CrossOverGPS, after thumbing through numerous user guide books and online stores. I believe I decided to purchase this unit after reading the review by GPS Magazine on Dec 11 (timestamp is provided by del.icio.us).

This unit met several of my criteria:

1. SiRF III chipset.
2. Tracking
3. Turn-by-turn voice directions with street name
4. Custom POIs.
   

[Post picture]

Who cares about integrating GPS with data network? I'll wait until someone fix the coolest hack of 2007.

And I receive my unit today, I could have had it yesterday, but I was out of commission and couldn't function half of the day.

The problem with this unit is immediately noticed when it is fired up: satellite lock is taking too long (~5 min on open air, impossible in the building).

Then the unit presents me with 6 applications:,

1. Vehicle Nav
2. Marine
3. Outdoor
4. System
5. Music
6. Photos
   

Since I will be sitting in a vehicle most of the time when using the unit, I give Vehicle Nav a try. The review by GPS Magazine is correct, it mentioned the unit's processor being under-powered. Indeed, the application takes a while to be fully loaded.

First, try the street address, it could search a street by city and zip code. The strong point of using this unit: it eliminates wrong entry by disabling the key and auto complete features.

Taking the unit to the street, the rerouting calculation is prompt, other people give bad reviews on the re-routing recalculation time, but I don't have a problem with it. Maybe it is easier to detour in Southern California?

On the way home, I turned on the tracking feature via outdoor application. The resulting log was exported to SD card, converted to KML format using Perl, and uploaded to Google Map.
Checking the map, I found that it is fairly accurate, since it tracked the movement of the car at the correct lane of 605 freeway (north bound instead of south bound or out of the freeway).

I couldn't wait to take the unit to the trails and uploading the results.

Out Of Order

Sick

Still I dragged myself to the office since I had meeting in the afternoon - How swell.

Groggy and very unproductive today....

Wicked Cool Perl Scripts

Wicked Cool™ Perl Scripts
Useful Perl Scripts That Solve Difficult Problems
by Steve Oualline

February 2006, 336 pp.
ISBN-10 1-59327-062-3
ISBN-13 978-1-59327-062-9

http://nostarch.com/frameset.php?startat=wcps

I like the book at the first glance since the TOC is fairly simple. The scripts are categorized into several area, such as Unix System Administration and Internet Data Mining, per chapter.

Flipping to the next page, I found the detailed table of content, listing the short description of the scripts and sub-sections. The sub-sections describes the usage/background/possible improvement after the code is given. I like the layout for each script since it mimics how POD supposed to be written.

But looking into each of the scripts, I found most of the scripts would save me a lot of time and agony ... 4 years ago. Thus, I would encourage the younglings to read this book ASAP before they start building their own library.

These are the general "must have" scripts:

• #2 Finding Duplicate Files: generally this is must have scripts, especially for someone who has amassed tons of media (aka p0rn) files and they aren't managed very well. You'll be surprised.
• #3 Checking for Changed Files: secure conscious user will find this script useful.
  
• #6 Website Link Checker
• #7 Orphan File Checker
• #19 Comic Download: the script is fairly big, but no fret since the code is available for download at nostarch.com. Comic lovers will be glad to have this script running on a regular basis.
  
• #22 Checking Symbolic Links
• #23 Disk Space Alarm
• #37 Dead Code Locator
• #38 EOL Type Detector
  

Besides those general scripts above, I believe some people will find that they need the others since they could be wearing a different hat than I am, for example: a CGI developer could find the CGI Debugging chapter very useful (depending on the level of proficiency).

These are the scripts that I find interesting since I am playing with map caching nowadays:

• #40 Getting the Map
• #41 Map Generator
• #42 Location Finder

Although it is a useful script, I couldn't help myself to point out that using Filesys::DiskSpace module for #23 Disk Space Alarm script is overflowing the brain with 'not-very-useful' and obscured module. Why not parsing the output of df command instead?

I am looking forward to the second edition to this book, giving out more goodies to the readers at large.

New Year Resolution

With the craziness at the closing of 2007, this topic is:
[under construction]

Linux Server In A Stick, Second Trial

Looks like I screwed up the image when I tried to install the NDISWrapper since the stick doesn't reboot properly anymore.

I redeploy the image again to the USB drive ...maybe I need to invest on re-image script...

Visit Pasadena Public Library

I reconnect myself with Pasadena Public Library again, after being absent for more than 2 years.

It is nice to be in the great hall again, waiting in the check out line. But WTH? My library card was having a problem, it was referring the holder of the card as someone named Matt.

Install NDISWrapper For Wireless Access

I attempt to install NDISWrapper for wireless access following this instruction: http://ubuntuforums.org/showthread.php?t=201902.

Apparently something is not right, since it doesn't work.

Godiva Goodies

The eee PC arrived today, it was small and cute! I managed to play with it for 2 hours, but I decided that I should probably buy N800 instead.

My dad and Lisa were fighting over presents from Ida, so funny. But she managed to walked away with a pack of biscotti. Beside the Godivas, he confiscated most of the chocolate supplies in the house and took it with him back to Jakarta.

We arrived at LAX around 8PM for 11PM flight, and the check-in process was great since the it wasn't crowded.

The First Day of 2008

Ida's flight departed at 12 from BUR. It was pretty hard to see her away.

We (the whole family and Stella) went to lunch together at Lucky City and spent the rest of the day at home catching up the latest news from Stella.

I closed the day by moaning the fact that I had to work tomorrow, after 11 days of being not productive.

Last Day of 2007

I spent the last day of 2007 driving through OC, and looking at the new properties. Apparently the prices still hold and I believe the housing market in OC hasn't tanked enough. We'll see what will happen in 2008.

After dinner, we went to Colorado Blvd. taunting the people who waited along the street for the 2008 Rose Parade, and we got tortillas, marshmallows and shaving cream treatment by the indignant people.

I sent Ida and Irwin back to Hollywood after we finished the bubbles.