| View previous topic :: View next topic |
| Author |
Message |
misi01
Advanced
Joined: 02 Dec 2002
Posts: 629
Topics: 176
Location: Stockholm, Sweden
|
 Posted: Wed Dec 02, 2015 6:46 am Post subject: XML Parse and restartability |
 |
|
We obviously have generalized programs in place in order to be able to restart a BMP halfway through its processing. This involves calling the general program to take checkpoints regularly (mainly to release locks on DB2 tables).
Using GSAM, we can be assured that IMS will position any read pointers to the relevant rows in any files. This works fine using "ordinary" files. Now, to the crunch. In the example that's causing a problem, we're reading an XML file. Let's assume it looks something like this:-
| Code: |
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Skatteverket
xmlns="http://xmls.skatteverket.se/se/skatteverket/ai/instans/infoForBeskattning/1.2"
xmlns:gm="http://xmls.skatteverket.se/se/skatteverket/ai/gemensamt/infoForBeskattning/1.2"
xmlns:ku="http://xmls.skatteverket.se/se/skatteverket/ai/komponent/infoForBeskattning/1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" omrade=""
xsi:schemaLocation="http://xmls.skatteverket.se/se/skatteverket/ai/instans/infoForBeskattning/1.2
http://xmlsutv.skatteverket.se/schemalager/se/skatteverket/ai/kontrolluppgift/instans/Kontrolluppgifter_1.2.xsd">
<ku:Avsandare>
various records here
</ku:Avsandare>
<ku:Blankettgemensamt>
various records here
</ku:Blankettgemensamt>
|
and now come the actual documents
| Code: |
<ku:Blankett nummer="2322">
various records
</ku:Blankett>
<ku:Blankett nummer="2322">
various records
</ku:Blankett> <------ checkpoint taken here
<ku:Blankett nummer="2322"> <------- Document nr 3
various records
</ku:Blankett>
etc etc
|
Let's assume the program crashed after the checkpoint was taken above. When we restart the program, the first record read will be the Document nr 3 (as expected).
Trouble is, according to our DBA, the XML Parser won't work properly since it's missing the "starting records" in the XML string. (this according to the DBA)
Has anyone experienced this sort of behaviour? Is there any way round it other than simply reading the file from the very beginning ?
_________________
Michael |
|
| Back to top |
|
 |
kolusu
Site Admin
Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose
|
| Posted: Wed Dec 02, 2015 10:10 am Post subject: Re: XML Parse and restartability |
|
|
| misi01 wrote: |
Let's assume the program crashed after the checkpoint was taken above. When we restart the program, the first record read will be the Document nr 3 (as expected).
Trouble is, according to our DBA, the XML Parser won't work properly since it's missing the "starting records" in the XML string. (this according to the DBA)
Has anyone experienced this sort of behaviour? Is there any way round it other than simply reading the file from the very beginning ? |
Misi01,
Unless I am missing something obvious, why can't you simply save the first 3 lines in the Checkpoint area and upon restart read those values once again and start off where you crashed.
_________________
Kolusu
www.linkedin.com/in/kolusu |
|
| Back to top |
|
|
kolusu
Site Admin
Joined: 26 Nov 2002
Posts: 12376
Topics: 75
Location: San Jose
|
|
| Back to top |
|
|
William Collins
Supermod
Joined: 03 Jun 2012
Posts: 437
Topics: 0
|
| Posted: Thu Dec 03, 2015 2:16 am Post subject: |
|
|
| Since XML can be presented to you as one big lump or one element at a time or something in between, the PARSE allows processing for that. You never have to build a complete XML document before you are able to PARSE it. |
|
| Back to top |
|
|
misi01
Advanced
Joined: 02 Dec 2002
Posts: 629
Topics: 176
Location: Stockholm, Sweden
|
| Posted: Sat Dec 05, 2015 4:17 am Post subject: |
|
|
Thanks to both.I'll have to keep testing and see what happens.
Kolusu - your idea of removing the headers and running it as a simple QSAM input file is a good idea.I'll see what it produces and get back
_________________
Michael |
|
| Back to top |
|
|
misi01
Advanced
Joined: 02 Dec 2002
Posts: 629
Topics: 176
Location: Stockholm, Sweden
|
| Posted: Thu Jul 07, 2016 4:23 am Post subject: Thought I'd append an update to my original question |
|
|
Kolusu - you were correct. I'll append an example and the code I used to be able to restart using GSAM as an input file. First of all, the file's root element
| Quote: |
<?xml version="1.0" encoding="utf-8"?>
<ns1:InstrumentInfo xmlns:ns0="xxx.xxx.Types.v1" xmlns:ns1="xxx.xx.Instrument
<ns1:TargetSystem>INETExt</ns1:TargetSystem>
<ns1:FileTimestamp>2015-10-16T11:17:13</ns1:FileTimestamp>
<ns1:NoOfInstruments>13738</ns1:NoOfInstruments>
<ns1:InstrumentDataList>
|
Then each element starts with the following:-
| Quote: |
<ns1:InstrumentData>
<ns1:VpdbId>2</ns1:VpdbId>
<ns1:InstrumentIdentificationList>
<ns0:InstrumentIdentification xsi:type="ns0:InstrumentId
<ns0:InstrumentIdType>VPDB Id</ns0:InstrumentIdType>
Etc etc
|
My code to read the GSAM file contains the following:-
| Code: |
if vx-start-of-document-read-no
perform xma-append-start-of-document
end-if
|
And
| Code: |
*****************************************************************
* Append all the records "belonging" to the start-of-document
* to aa-start-of-document
*****************************************************************
xma-append-start-of-document section.
*
move aa-shbg0910-post-lgd to ix-1
compute ix-2 = ix-1
+ ix-start-of-document
*
evaluate true
when ix-2 > length of aa-start-of-document
* The length of aa-start-of-document is too small
* to receive the complete root element. Tell them
* and abend
move kk-al-01 to shbgg015-avbrottslage
move kk-er to shbgg015-returkod
string
'aa-start-of-document is defined with too few '
'characters to receive ALL of the START-OF-DOCUMENT. '
'Increase (if possible) and try again'
delimited by size
into shbgg015-meddelande-tab
end-string
perform xf-user-snap
perform xg-abend
when other
string
aa-shbg0910-post-vardata(1:ix-1)
delimited by size
into aa-start-of-document
with pointer ix-start-of-document
end-string
end-evaluate
*
exit.
|
At the start of the program, we call a module to check whether we’re running “normally”
or restarting after an abend
| Code: |
if vanlig-korning
* Normal scenario move 1 to ix-start-of-document
perform xm-las-gsam-n022201
else
* Restart
move ' ' to aa-shbg0910-post-vardata
* Pretend we read all the starting records and catenated
* then into ONE record
move aa-start-of-document to aa-shbg0910-post-vardata
move ix-start-of-document to aa-shbg0910-post-lgd
end-if
|
And finally, the actual XML PARSE
| Code: |
xml
parse aa-shbg0910-post-vardata(1:aa-shbg0910-post-lgd)
processing procedure cb-xml-handler
on exception
perform ca-hantera-parse-exception
end-xml
|
Have to append this as-is, since there're obviously some UTF characters somewhere in it so I can't preview it.
_________________
Michael |
|
| Back to top |
|
|
|
|
|
FAQ
Search
Quick Manuals
Register
Profile
Log in to check your private messages
Log in
