MVSFORUMS.com

[jsharon1248]

There's no debate that loading/retrieving data within an internal table will outperform any IO. But that would require modifications to the existing application architecture. Looking at the biggest bang for the buck, there will be lower risk and effort (cost) to attempt relatively simple modifications that do not impact the existing application architecture. If changing the CI size and increasing buffers reduces the run times to acceptable levels, great. If not, there's little loss.

Also, referring to the suspected problem file, hellohatim stated:

[hellohatim]

Hi Everybody,

Thanks once again for all your inputs.

The KSDS file does have 99,999 records only. the 6.9 M statistic was # of hits on KSDS not the number of records. The input sequential file is having > 7M records and for each record from this file, the zip code variable is used to index into the KSDS file.

[Terry_Heinze]

Can you post a LISTCAT of the KSDS file after it's been loaded?
_________________
....Terry

[hellohatim]

Hi,

Please find the Listcat output attached.

Regards,
Hatim.

[Terry_Heinze]

Well, it's obvious that CI splits or CA splits are not the problem! I'm no VSAM expert, but BUFSPACE seems a bit small. Also, aren't the usual SHROPTNS (2,3) instead of (3,3)? For random access, DO NOT use DYNAMIC access, use RANDOM access. Also, random access files can benefit significantly from using BLSR.
_________________
....Terry

[jsharon1248]

Reviewing the LISTCAT, I noticed that you're allocating over 100 times more space than you're using. Data HI-A-RBA is 314081280 but HI-U-RBA is only 1474560. Index HI-A-RBA is 678912 but HI-U-RBA is only 4608. I know space is cheap but that's over 400 CYLS that can't be used for anything else. You could easily get by with an allocation of CYLINDERS(5 1).

If you decide to experiment with BUFNx parms, I would initially leave the data CI size as is (4096) and try BUFNI=3. For this file, there are only 3 index CI's; 1 root CI and 2 leaf CI's. The problem is that the default buffers allocated for VSAM are 1 index buffer and 2 data buffers. For every read, you're swapping out the root CI and one of the leaf CI's. That's 2 physical IO's for every logical read just for the index. Then add 1 physical IO if the data CI isn't in one of the two data buffers. Using BUFNI=3, you'll store the entire index in the buffers. For the BUFND value, I suspect that the nature of the input data results in something close to skip sequential processing. Most likely, there are pockets of records on the input file where the zip values are similar. If that's the case, leave the data CI size alone and try BUFND=10 or even 20. If that's not the case and the distribution really is random, you could reduce the data CI size to 512 and get by with BUFND=5. Changing the data CI size would also result in a change to the index CI size and you'd want to increase the BUFNI value to number of index CI's. If you see good results playing around with the BUFNx parms for this file, you might want to consider experimenting with these parms for the other VSAM files too. Just for fun.

By the way, based on the interest generated by your post, you can tell what excites the masses. Thanks for the fun.

[dbzTHEdinosauer]

jsharon1248,

great input. as you indicated, this file is so small that with the right allocaton for the index and data, it could be contained completely in memory, thus no application logic change.

I have always found it easier to just load files into COBOL tables since I could never get the systems types to properly allocate the vsam structures.

The other three files: PLANIND, TM, and ZIPDMA might also receive performance gains if they were allocated differently.

Does not matter much if the file is in memory or a COBOL table - get rid of the physical I/O's.
_________________
Dick Brenholtz
American living in Varel, Germany

[hellohatim]

jsharon1248 & Dick,

Thanks a lot for your inputs. Yes this one was a major flaw, have been allocating undue space to the zipcode files. I will definitely tune the VSAM files for various parameters.

One more thing, we merged all three zip code files, TM, DMA, SMSA into one zip file with three columns. We have observed four times reduction in the job cost. This one is actually a design flaw, we could have done with just one file right from the begining

I will post the results by next week for suggestions posted by jsharon1248.

Thanks & Regards,
Hatim.
_________________
-Hatim M P

[jsharon1248]

Hatim

Thanks for the followup and looking forward to the results next week.


One more 'by the way'. I'm heading straight for the 'About MVSForums' to start a new thread to praise this site. I'm a newbie, but I've been around long enough to recognize exceptional quality.

[hellohatim]

Hello,

Sorry for the delayed response. Please find the comparisons below...

[hellohatim]

Hi,

I did run the original production job with 4 VSAM file but this time with the following parameters...

[jsharon1248]

A few things. First, I don't see a cost for the first 2 runs. If we're going to make meaningful comparisons, we need to see the initial costs. Second, I don't know what the counts represent. I think one of them is EXCP's, but I'm not sure. I'd also like to see run times, a LISTCAT, and the actual BUFNx values used. From what I see, there is a downward trend. The last 3 runs show significant improvements over the original. As expected, the internal WS tables show the best results, but merging the files into 1 KSDS is pretty close.

[jsharon1248]

I'm sorry, I missed your last post. I think you're way too high on the BUFNI values. Send a LISTCAT and we can see if we can squeeze a little more out of this.

[hellohatim]

Hi jsharon1248,

The listcat dumps are pretty big.Can you please let me know your email id so that I can mail you the dumps? Or if you need only some of the stats, I can copy paste them here.

I did not include the cost of other two jobs becuase they included other steps as well. The statistic I had provided was only for STEP0090, the one which had VSAM file issue. Remaining jobs I had run with STEP0090 only for testing. I will send the elapsed times later as well.

Taking my last post as example, 21447K is EXCP count and 12.96 CPU Time, i.e 5 & 7 column respectively.

Regards,
Hatim.
_________________
-Hatim M P

[jsharon1248]

I'd only want to see the values for BUFSPACE, CISIZE, REC-TOTAL,HI-A-RBA,HI-U-RBA, SPACE-PRI for the DATA and INDEX sections. Also, LEVELS from the INDEX section. I don't think we'll seen any drastic improvements, but there could still be slight gains.