Posted: Thu Nov 20, 2003 9:36 am Post subject: How can we do this using Sort..?
Hi,
I have a job that runs every business day and creates a new generation of a GDG(the GDG base has been created with limit 180 and we are not allowed to change this).
The first field in this file is the current date(ie; the date when the new generation is created.Format is YYYYMMDD)
Now my requirement is to automate a monthly job that runs on 1st of every month (Before the daily job runs.ie the current generation will be created on the last business day of the previous month).The input for this monthly job are the files that were created in the previous month.Since we can't predict the number files that will get created each month it's not possible to hardcode the generations(ie:0,-1,-2 ...etc).So by specifying the GDG base as the input file(this will take all the 180 generations) how can we extract only the required records for the monthly job.? To be more specific if I'm running my monthly job on 1st of Dec 2003
I have to capture all the records from the respective files that were created on the month of November 2003.
Appreciate any help on this.. _________________ Regds,
Somu
Joined: 15 Dec 2002 Posts: 637 Topics: 43 Location: Bengaluru, INDIA
Posted: Thu Nov 20, 2003 10:02 am Post subject:
Quote:
The first field in this file is the current date(ie; the date when the new generation is created.Format is YYYYMMDD)
So, this means, in a given generation of a GDG, the first 8-bytes in ALL records will be the date field? If yes, then you can have INCLUDE/OMIT logic (using DFSORT) to filter out the records depending on the month.
For example, if the above assumption is true, then for the input as GDG base, you can have the SYSIN as,
Code:
//SYSIN DD *
OPTION COPY
INCLUDE COND=(1,4,CH,EQ,C'2003',&,
5,2,CH,EQ,C'11')
/*
Or, is something else the requirement? _________________ ALL opinions are welcome.
Debugging tip:
When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.
-- Sherlock Holmes.
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Thu Nov 20, 2003 10:07 am Post subject:
Cogito-Ergo-Sum,
Somu needs to automate the job every month , so he just cannot keep changing the include cond every month. We need to generate the include cond dynamically every time the job runs. I will post the solution in a few mins
Joined: 02 Dec 2002 Posts: 310 Topics: 8 Location: Texas
Posted: Thu Nov 20, 2003 10:33 am Post subject:
It's not uncommon to have daily, monthly,yearly or whatever gdgs that rollup to a higher level to prevent your problem. See if they'll let you use a daily and monthly set of gdgs. On the 1st of every month copy all your daily gdgs to a monthly gdg and then wipeout your daily gdg and start over again on daily gdg 1.
You job then simply calls for the monthly gdg. There can be several variations on how to do this. Set your monthly gdg limit to 6 and you'll still have the same data retention as 180 dailies gives you.
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Thu Nov 20, 2003 10:59 am Post subject:
Somu,
The following DFSORT/ICETOOL JCL will give you the desired results.If you have syncsort at your shop then change the pgm name to synctool.
I am posting 2 versions of the same job. If you don't have the latest version of DFSORT or syncsort use solution2.
Solution1: A brief description of the solution. The objective of this job is to create a dynamic control card for the copy process. We use the date parm (DATE1(YYYYMMDD)) on the inrec fields and using a change on outrec fields we will manipulate the year. This change command will only come into effect only when the month is january. In that case we need to subtract 1 year from the current year.so we coded a change command to decrement the year by 1.Similary we use another change command to decrement the month by 1. Once we have decremented the year and month we generate a include cond as follows.
Code:
INCLUDE COND=(1,6,CH,EQ,C'200310')
Now using this as a control card we copy all the records from the gdg generations.
Solution2:
If the previous solution gives you an error on DATE1 parm, then you have a older version of the sort product. In that case use the following solution. This solution is also similar to the solution1 except that we need to use the reporting features to get the current date.
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Thu Nov 20, 2003 11:16 am Post subject:
Here's an easier way to do this with DFSORT:
Code:
//S1 EXEC PGM=ICEMAN
//SYSOUT DD SYSOUT=*
//SORTIN DD *
DUMMY
/*
//S1 DD DSN=&&S1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//S2 DD DSN=&&S2,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//SYSIN DD *
OPTION COPY
* Create C'yyyymm' constant for current date
INREC FIELDS=(DATE2)
* Create DFSORT symbol for previous month as:
* LAST_MON,+aaaabb
*
* If mm is not 01, set bb = mm - 1
OUTFIL FNAMES=S1,
INCLUDE=(5,2,CH,NE,C'01'),
OUTREC=(C'LAST_MON,+',
1,4, yyyy
5,2,ZD,SUB,+1,TO=ZD,LENGTH=2, mm - 1
80:X)
* If mm is 01, set aaaa = yyyy - 1 and set bb = 12
OUTFIL FNAMES=S2,
SAVE,
OUTREC=(C'LAST_MON,+',
1,4,ZD,SUB,+1,TO=ZD,LENGTH=4, yyyy - 1
C'12', 12
80:X)
/*
//S2 EXEC PGM=ICEMAN
//SYMNAMES DD DSN=&&S1,DISP=(OLD,PASS)
// DD DSN=&&S2,DISP=(OLD,PASS)
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=... input file
//SORTOUT DD DSN=... output file
//SYSIN DD *
OPTION COPY
* Use LAST_MON symbol in INCLUDE statement
INCLUDE COND=(1,6,ZD,EQ,LAST_MON)
/*
_________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Thu Nov 20, 2003 11:35 am Post subject:
Somu,
Here is another solution. This solution uses the horizontal math functions to generate the dynamic control cards.The following DFSORT/ICETOOL JCL will give you the desired results.
A brief explanation of the JOb. We use the DATE1 parm to get the current date in the form of YYYYMMDD. Taking this record we create 2 temp files. If the month is 1 then we create C1 and for all others we create C2.
If the current month is January (01), then we subtract 1 from the year using horizontal math function.we hardcode a value of 12 for this case.
If the month is any thing other than january (01) we just subtract 1 from the month part.
So at any given time we will either create C1 or C2 depending on the month.
We now concatenate these 2 temp datasets (one would empty) and copy the last month records.
Joined: 02 Dec 2002 Posts: 1618 Topics: 31 Location: San Jose
Posted: Thu Nov 20, 2003 11:40 am Post subject:
Kolusu,
Looks like we came up with two different variations of the same idea (great minds "sink" in the same channels) _________________ Frank Yaeger - DFSORT Development Team (IBM)
Specialties: JOINKEYS, FINDREP, WHEN=GROUP, ICETOOL, Symbols, Migration
DFSORT is on the Web at:
www.ibm.com/storage/dfsort
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Thu Nov 20, 2003 11:44 am Post subject:
Frank,
I had the topic open and I just posted the solution after thinking about it .I just did not hit refresh and posted the solution. When I hit submit button then only ,I noticed that you posted your solution. As Always Master(Frank) Rules !!
Joined: 02 Dec 2002 Posts: 310 Topics: 8 Location: Texas
Posted: Thu Nov 20, 2003 11:50 am Post subject:
Another way would be to parse the results of a listcat on the gdg base and build a list of gdg dsnames that were created in the previous month. Use that to construct the jcl needed and submit the job.
It just seems a waste to be to read every record in a file to check a date when that field should have the same value in every record in that gdg. Unless my IEYEBALL scan of the posted solutions is wrong. Also unless these are huge files then who cares.
Joined: 26 Nov 2002 Posts: 12376 Topics: 75 Location: San Jose
Posted: Thu Nov 20, 2003 12:07 pm Post subject:
Taltyman,
There is only one problem using listcat info. The creation date of listcat info will be in Julian format. for ex: 2003.322.
Now we need to get an internal table of julian date for the first of every month i.e
2003.001 - January
2003.032 - Feb..
...
....
Now we need to validate the creation dates with 2 boundaries. One with last month's first date and another with current month's first date.
I am not an expert in rexx, but I am very much interested to know if we can generate the internal table with julian date.
The other option is to convert all the GDG creation julian dates to gregorian date and take the GDG gen name.
But I still may have a problem. Let us think of this scenario.
The gdg creation job runs at 11 pm every day. On the 31st of october, this job kicked off at 11 pm and for some reason completed at 1 am . Now the creation date for this gdg would be the 1st of nov. But this file actually contains the data of october.So if we parse the listcat info we would definitely skip this gen which is not correct.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum