Joined: 15 Dec 2002 Posts: 637 Topics: 43 Location: Bengaluru, INDIA
Posted: Fri Apr 14, 2006 6:14 am Post subject: Record deleted from output file when program restarted !
Hi,
I have an output file which is populated from the messages (using MQGET) in a MQ queue. To test checkpoint/restart, I have adjusted the program such that after every record is written, I am issuing a MQCOMMIT followed by a checkpoint . After three such iterations, I am doing a divide-by-zero operation to force an abend.
When the program abends, spool shows two checkpoints and output file is having 3 records. I am restarting my program with checkpoint ID as 2 and file allocated as DISP=MOD.
During restart, the remaining messages from queue is written, but, the third record (in the normal run) is deleted !
I am at a loss to understand how this is possible. Help ! _________________ ALL opinions are welcome.
Debugging tip:
When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.
-- Sherlock Holmes.
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Fri Apr 14, 2006 7:47 am Post subject:
Quote:
When the program abends, spool shows two checkpoints and output file is having 3 records. I am restarting my program with checkpoint ID as 2 and file allocated as DISP=MOD.
During restart, the remaining messages from queue is written, but, the third record (in the normal run) is deleted !
Cogito,
If I am not wrong your program has only taken 2 checkpoints. so IMS automatically sets the pointer to end of 2nd record. So theortically you should the 3rd record once again and re-process it. Had it been a GSAM file input IMS would have the set the pointer at 3rd record and you would have processed the 3rd record.
I suggest that you follow this procedure.ie. switch the order of checkpoint and mqcommit.
1. read the queue
2. write to output file
3. Take a checkpoint
4. Issue Mqcommit
Also I would suggest that you remove the Disp=MOD parm.
Joined: 15 Dec 2002 Posts: 637 Topics: 43 Location: Bengaluru, INDIA
Posted: Fri Apr 14, 2006 7:57 am Post subject:
Kolusu,
Yes, the program has taken only two checkpoints. I am still not sure, why it is so. (Part of the reason why I do not know is, the code for DLI calls have been 'shielded' from us and we just call the right sub-program. I am yet to dig out the program that does this.)
Quote:
So theortically you should the 3rd record once again and re-process it.
Did you mean, my output file should have duplicated the third record? If yes, then I was thinking the same too.
Any particular reason, for swapping MQCOMMIT and checkpoint ? Suppose there is an abend in the program between MQCOMMIT and checkpoint, then the record which got checkpointed, is still there in the queue. Wouldn't a restart duplicatet the output ?
What do I replace DISP=MOD with ? If I code DISP=OLD, then the output would have only the subsequent messages. But, I want all. _________________ ALL opinions are welcome.
Debugging tip:
When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.
-- Sherlock Holmes.
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Fri Apr 14, 2006 8:11 am Post subject:
Quote:
Did you mean, my output file should have duplicated the third record? If yes, then I was thinking the same too.
Any particular reason, for swapping MQCOMMIT and checkpoint ? Suppose there is an abend in the program between MQCOMMIT and checkpoint, then the record which got checkpointed, is still there in the queue. Wouldn't a restart duplicatet the output ?
What do I replace DISP=MOD with ? If I code DISP=OLD, then the output would have only the subsequent messages. But, I want all.
NO the 3rd record will not duplicated. IMS automatically keeps track of the records it has written. In Case of the program abend the file Pointers will go back to the last checkpoint record written. Even if your file has the 3rd record, IMS sets the pointer after the 2nd record. So if you had re-processed the 3rd input record it would have overlaid the existing 3rd record in the output file.
I wanted to switch the MQCOMMIT and CHECKPOINT to make sure that record is written. But as you mentioned this would result in a duplicate row if the program abended in between checkpoint and Mqcommit.
You can avoid that by storing the key you have written in the checkpoint area. In case of restart you simply check if the key you are about write is the same key as the checkpoint key. If it is the same then skip and move to the next record.
Coding DISP=OLD does not make you loose the data. As IMS has the ability to set the file pointers to the last checpointed record.
Joined: 15 Dec 2002 Posts: 637 Topics: 43 Location: Bengaluru, INDIA
Posted: Mon Apr 17, 2006 10:21 am Post subject:
Kolusu,
Thank you for the inputs.
I put some more testcases and I understood the results of all; execpt the following one.
First, the program would do a MQGET to get the message. Next, it will issue a ISRT call to write that message to an output file. Then, the program does a MQCOMMIT followed by a CHKP.
I have 5 messages in the queue and setup the program such that, the MQCOMMIT and CHKP are done once every 2 messages. I made my program to abend (divide by zero option) when the last message has been read and written to file and MQCOMMIT is done.
In the spool, I see highest CHKPTID as 2 (which is probably ok), queue is empty, file has 5 records. When I restarted with CHKPTID as 2, and file as DISP=MOD, I see the last record deleted ! That is, O/P file is having only 4 records.
How is this possible ? _________________ ALL opinions are welcome.
Debugging tip:
When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.
-- Sherlock Holmes.
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Mon Apr 17, 2006 4:39 pm Post subject:
Quote:
file has 5 records. When I restarted with CHKPTID as 2, and file as DISP=MOD, I see the last record deleted ! That is, O/P file is having only 4 records.
How is this possible ?
As I said earlier IMS set the pointer to the end of the 4th record(2 check points) and after that there are no records written and when the program ended , the file is closed and IMS writes the EOF mark at the end of 4th record there by deleting your 5th record.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
FAQ
Search
Quick Manuals
Register
Profile
Log in to check your private messages
Log in
