VSAM Performance - CPU reduction

Discussion:

(too old to reply)

Arun Venkatratnam

2018-01-06 00:28:48 UTC

Hi All,

We are looking to improve the performance of a COBOL program that processes 2 VSAM files. The first file is the I/P file and every record read from the input VSAM file is searched for a matching record in the other file and a report is written. The program also applies some business rules while comparing each matching record.

The I/P file is read sequentially while the other file is read in a skip sequential basis. The test files that were used had 32M records each while production files have 110M records each.

Attached is the strobe report from the execution of the test job. The test job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU minute of execution time.

We are attempting to optimize the VSAM access to these files as it is seen to take more than 50% of the CPU consumed by this job.

In the 'Attribution of CPU execution time' section, we see that the major contributors are the components 'QSAM INIT I/O & EXITS' (Module IGZEQBL) and PARTITION COMMUNICATION.

Could you please help us understand:

1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.

Thank you

Arun

------------------------------------------------------------------------------------------------------------------------------------------------------------------

1Strobe* PERFORMANCE PROFILE PROGRAMA 01/02/2018 PAGE 42

- #ACE ** ATTRIBUTION OF CPU EXECUTION TIME **
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY--------------------- -----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE SECTION DESCRIPTION SOLO TOTAL

.LELIB CEEBINIT LE/370 BATCH INIT/TERM .32 .32
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL QSAM INIT I/O & EXITS 1.93 1.93

XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE SECTION DESCRIPTION

PROGRAMA PROGRAMA 003522 1.30 1.30
----- -----
3.55 3.55
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY--------------------- -----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE SECTION DESCRIPTION SOLO TOTAL

.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL QSAM INIT I/O & EXITS 1.84 1.84
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL CURRMEM QSAM INIT I/O & EXITS 4.34 4.38
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL DVFILE QSAM INIT I/O & EXITS .03 .03
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL PREVMEM QSAM INIT I/O & EXITS 22.48 22.51

XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE SECTION DESCRIPTION

PROGRAMA PROGRAMA 003522 IGZCPCO CURRMEM PARTITION COMMUNICATION 4.15 4.19
PROGRAMA PROGRAMA 003522 IGZCPCO IGZEVIO VSAM INPUT/OUTPUT .57 .57
PROGRAMA PROGRAMA 003522 IGZCPCO PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----
50.22 50.32

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Lizette Koehler

2018-01-06 00:44:51 UTC

Permalink

Hello -

You might be seeing issues with physical dasd retrieval of the data. But that is just a guess.

Have you looked at adding STRNO on the VSAM definition or using RLS to provide the records to be instorage?

In your JCL for the job, have you coded BUFND, or BUFNI or BUFSP?

Could you provide the LISTC ENT(vsam-name-here) ALL for us to review?

What are your share options?

When was the Cobol program last compiled and lked?

What version of Cobol was being used when it was last compiled and lked?

Has a review of the Cobol program logic been done to see if any performance can be gotten there?

What version of z/OS are you running on?

Are there any other processes using this VSAM dataset when it is being accessed by this program? DB2, CICS, etc....

Lizette

-----Original Message-----
Behalf Of Arun Venkatratnam
Sent: Friday, January 05, 2018 5:20 PM
Subject: VSAM Performance - CPU reduction
Hi All,
We are looking to improve the performance of a COBOL program that processes 2
VSAM files. The first file is the I/P file and every record read from the
input VSAM file is searched for a matching record in the other file and a
report is written. The program also applies some business rules while
comparing each matching record.
The I/P file is read sequentially while the other file is read in a skip
sequential basis. The test files that were used had 32M records each while
production files have 110M records each.
Attached is the strobe report from the execution of the test job. The test
job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU minute
of execution time.
We are attempting to optimize the VSAM access to these files as it is seen to
take more than 50% of the CPU consumed by this job.
In the 'Attribution of CPU execution time' section, we see that the major
contributors are the components 'QSAM INIT I/O & EXITS' (Module IGZEQBL) and
PARTITION COMMUNICATION.
1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.
Thank you
Arun
-----------------------------------------------------------------------------
-----------------------------------------------------------------------------
--------
1Strobe* PERFORMANCE PROFILE PROGRAMA
01/02/2018 PAGE 42
- #ACE ** ATTRIBUTION OF CPU EXECUTION TIME **
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY--------------------- -----------
------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE
SECTION DESCRIPTION SOLO TOTAL
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
.32 .32
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL
QSAM INIT I/O & EXITS 1.93 1.93
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE
SECTION DESCRIPTION
PROGRAMA PROGRAMA 003522
1.30 1.30
----- -----
3.55 3.55
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY--------------------- -----------
------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE
SECTION DESCRIPTION SOLO TOTAL
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL
QSAM INIT I/O & EXITS 1.84 1.84
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL
CURRMEM QSAM INIT I/O & EXITS 4.34 4.38
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL
DVFILE QSAM INIT I/O & EXITS .03 .03
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL
PREVMEM QSAM INIT I/O & EXITS 22.48 22.51
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE
SECTION DESCRIPTION
PROGRAMA PROGRAMA 003522 IGZCPCO
CURRMEM PARTITION COMMUNICATION 4.15 4.19
PROGRAMA PROGRAMA 003522 IGZCPCO
IGZEVIO VSAM INPUT/OUTPUT .57 .57
PROGRAMA PROGRAMA 003522 IGZCPCO
PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----
50.22 50.32
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to

Ron hawkins

2018-01-06 02:55:17 UTC

Permalink

Arun,

Strobe gives you a lot more information than this little tidbit.

I cannot answer your specific questions, but you mention that the 2nd dataset is skip sequential. Is there some chance that someone has set this up to be accessed as LSR with SMS or Batch/LSR?

If you can estimate the number of records that are being accessed sequentially, then you should be able to come up with a large CISZ and BUFND that will minimize the number of SSCH issued for each sequential access using NSR.

Ron

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Arun Venkatratnam
Sent: Friday, January 5, 2018 4:20 PM
To: IBM-***@LISTSERV.UA.EDU
Subject: [IBM-MAIN] VSAM Performance - CPU reduction

Hi All,

We are looking to improve the performance of a COBOL program that processes 2 VSAM files. The first file is the I/P file and every record read from the input VSAM file is searched for a matching record in the other file and a report is written. The program also applies some business rules while comparing each matching record.

The I/P file is read sequentially while the other file is read in a skip sequential basis. The test files that were used had 32M records each while production files have 110M records each.

Attached is the strobe report from the execution of the test job. The test job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU minute of execution time.

We are attempting to optimize the VSAM access to these files as it is seen to take more than 50% of the CPU consumed by this job.

In the 'Attribution of CPU execution time' section, we see that the major contributors are the components 'QSAM INIT I/O & EXITS' (Module IGZEQBL) and PARTITION COMMUNICATION.

Could you please help us understand:

1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.

Thank you

Arun

------------------------------------------------------------------------------------------------------------------------------------------------------------------

1Strobe* PERFORMANCE PROFILE PROGRAMA 01/02/2018 PAGE 42

- #ACE ** ATTRIBUTION OF CPU EXECUTION TIME **
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY--------------------- -----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE SECTION DESCRIPTION SOLO TOTAL

.LELIB CEEBINIT LE/370 BATCH INIT/TERM .32 .32
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL QSAM INIT I/O & EXITS 1.93 1.93

XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE SECTION DESCRIPTION

PROGRAMA PROGRAMA 003522 1.30 1.30
----- -----
3.55 3.55
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY--------------------- -----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE SECTION DESCRIPTION SOLO TOTAL

.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL QSAM INIT I/O & EXITS 1.84 1.84
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL CURRMEM QSAM INIT I/O & EXITS 4.34 4.38
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL DVFILE QSAM INIT I/O & EXITS .03 .03
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL PREVMEM QSAM INIT I/O & EXITS 22.48 22.51

XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE SECTION DESCRIPTION

PROGRAMA PROGRAMA 003522 IGZCPCO CURRMEM PARTITION COMMUNICATION 4.15 4.19
PROGRAMA PROGRAMA 003522 IGZCPCO IGZEVIO VSAM INPUT/OUTPUT .57 .57
PROGRAMA PROGRAMA 003522 IGZCPCO PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----
50.22 50.32

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Gerhard Adam

2018-01-06 03:14:10 UTC

Permalink

Seems a bit round-about.

Why not REPRO them to a sequential file, sort by key, and then produce your output.

Sent from my iPhone

Post by Arun Venkatratnam
Hi All,
We are looking to improve the performance of a COBOL program that processes 2 VSAM files. The first file is the I/P file and every record read from the input VSAM file is searched for a matching record in the other file and a report is written. The program also applies some business rules while comparing each matching record.
The I/P file is read sequentially while the other file is read in a skip sequential basis. The test files that were used had 32M records each while production files have 110M records each.
Attached is the strobe report from the execution of the test job. The test job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU minute of execution time.
We are attempting to optimize the VSAM access to these files as it is seen to take more than 50% of the CPU consumed by this job.
In the 'Attribution of CPU execution time' section, we see that the major contributors are the components 'QSAM INIT I/O & EXITS' (Module IGZEQBL) and PARTITION COMMUNICATION.
1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.
Thank you
Arun
------------------------------------------------------------------------------------------------------------------------------------------------------------------
1Strobe* PERFORMANCE PROFILE PROGRAMA 01/02/2018 PAGE 42
- #ACE ** ATTRIBUTION OF CPU EXECUTION TIME **
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY--------------------- -----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE SECTION DESCRIPTION SOLO TOTAL
.LELIB CEEBINIT LE/370 BATCH INIT/TERM .32 .32
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL QSAM INIT I/O & EXITS 1.93 1.93
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE SECTION DESCRIPTION
PROGRAMA PROGRAMA 003522 1.30 1.30
----- -----
3.55 3.55
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY--------------------- -----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE SECTION DESCRIPTION SOLO TOTAL
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL QSAM INIT I/O & EXITS 1.84 1.84
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL CURRMEM QSAM INIT I/O & EXITS 4.34 4.38
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL DVFILE QSAM INIT I/O & EXITS .03 .03
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL PREVMEM QSAM INIT I/O & EXITS 22.48 22.51
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE SECTION DESCRIPTION
PROGRAMA PROGRAMA 003522 IGZCPCO CURRMEM PARTITION COMMUNICATION 4.15 4.19
PROGRAMA PROGRAMA 003522 IGZCPCO IGZEVIO VSAM INPUT/OUTPUT .57 .57
PROGRAMA PROGRAMA 003522 IGZCPCO PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----
50.22 50.32
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

Wayne Bickerdike

2018-01-06 05:18:54 UTC

Permalink

Classic two file match. Sort the two and produce the report. Not a good use
of VSAM...

Post by Gerhard Adam
Seems a bit round-about.
Why not REPRO them to a sequential file, sort by key, and then produce your output.
Sent from my iPhone

On Jan 5, 2018, at 4:20 PM, Arun Venkatratnam <
Hi All,
We are looking to improve the performance of a COBOL program that

processes 2 VSAM files. The first file is the I/P file and every record
read from the input VSAM file is searched for a matching record in the
other file and a report is written. The program also applies some business
rules while comparing each matching record.

The I/P file is read sequentially while the other file is read in a skip

sequential basis. The test files that were used had 32M records each while
production files have 110M records each.

Attached is the strobe report from the execution of the test job. The

test job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU
minute of execution time.

We are attempting to optimize the VSAM access to these files as it is

seen to take more than 50% of the CPU consumed by this job.

In the 'Attribution of CPU execution time' section, we see that the

major contributors are the components 'QSAM INIT I/O & EXITS' (Module
IGZEQBL) and PARTITION COMMUNICATION.

1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these

components.

Thank you
Arun
------------------------------------------------------------

------------------------------------------------------------
------------------------------------------

1Strobe* PERFORMANCE PROFILE PROGRAMA

01/02/2018 PAGE 42

- #ACE ** ATTRIBUTION OF CPU

EXECUTION TIME **

-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

XACTION MODULE SECTION DESCRIPTION

MODULE SECTION DESCRIPTION SOLO TOTAL

.LELIB CEEBINIT LE/370 BATCH INIT/TERM

.32 .32

.LELIB CEEBINIT LE/370 BATCH INIT/TERM

IGZEQBL QSAM INIT I/O & EXITS 1.93 1.93

XACTION MODULE SECTION LOCATION LINE SOURCE TEXT

MODULE SECTION DESCRIPTION

PROGRAMA PROGRAMA 003522

1.30
1.30
----- -----
3.55 3.55

-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

XACTION MODULE SECTION DESCRIPTION

MODULE SECTION DESCRIPTION SOLO TOTAL

.LELIB CEEBINIT LE/370 BATCH INIT/TERM

IGZEQBL QSAM INIT I/O & EXITS 1.84 1.84

.LELIB CEEBINIT LE/370 BATCH INIT/TERM

IGZEQBL CURRMEM QSAM INIT I/O & EXITS 4.34 4.38

.LELIB CEEBINIT LE/370 BATCH INIT/TERM

IGZEQBL DVFILE QSAM INIT I/O & EXITS .03 .03

.LELIB CEEBINIT LE/370 BATCH INIT/TERM

IGZEQBL PREVMEM QSAM INIT I/O & EXITS 22.48 22.51

XACTION MODULE SECTION LOCATION LINE SOURCE TEXT

MODULE SECTION DESCRIPTION

PROGRAMA PROGRAMA 003522

IGZCPCO CURRMEM PARTITION COMMUNICATION 4.15 4.19

PROGRAMA PROGRAMA 003522

IGZCPCO IGZEVIO VSAM INPUT/OUTPUT .57 .57

PROGRAMA PROGRAMA 003522

IGZCPCO PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----
50.22 50.32

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

Arun Venkatratnam

2018-01-06 06:15:55 UTC

Permalink

Apologies. I missed to mention that depending on a record that matches in the 2nd file, there are some random reads that refer back to a previous record as per the business logic. Hence, the second file has to be VSAM. We also see a similar behavior with few other programs that do skip sequential reads on other VSAM files.

Thanks
Arun

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Arun Venkatratnam

2018-01-06 06:27:32 UTC

Permalink

Just to be clear. The similar behavior i am referring to is the strobe report listing QSAM INIT I/O & EXITS and PARTITION COMMUNICATION as the top CPU contributors.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Lizette Koehler

2018-01-06 15:09:22 UTC

Permalink

Sometimes I find Strobe reports misleading when reviewing them

They do a great job in showing how the process is working. But performance analysis takes a lot of experience to be able to use them well.

QSAM is Queued Sequential Access Method - it is what is used to read your data and return it to your program for action.

What I think you should do is to look at different areas you can control with your JCL and program

You might be seeing issues with physical dasd retrieval of the data. But that is just a guess.

Have you looked at adding STRNO on the VSAM definition or using RLS to provide the records to be instorage?

In your JCL for the job, have you coded BUFND, or BUFNI or BUFSP?

Could you provide the LISTC ENT(vsam-name-here) ALL for us to review?

Is the File KSDS, ESD, or Does it have INDEX/PATH/AIX?

What are your share options?

When was the Cobol program last compiled and lked?

What version of Cobol was being used when it was last compiled and lked?

Has a review of the Cobol program logic been done to see if any performance can be gotten there?

What version of z/OS are you running on?

Are there any other processes using this VSAM dataset when it is being accessed by this program? DB2, CICS, etc....

Have you reviewed RMF Data to see if the DASD was performing well? Or if the program was preforming well

The Strobe report will not be the final decision on what is going on with this process. RMF, JCL analysis, Program Analysis are all required to determine how to improve your process.

Lizette

-----Original Message-----
Behalf Of Arun Venkatratnam
Sent: Friday, January 05, 2018 11:29 PM
Subject: Re: VSAM Performance - CPU reduction
Just to be clear. The similar behavior i am referring to is the strobe report
listing QSAM INIT I/O & EXITS and PARTITION COMMUNICATION as the top CPU
contributors.
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to

Arun Venkatratnam

2018-01-06 17:24:51 UTC

Permalink

DATASET-OWNER-----(NULL) CREATION--------2018.005
RELEASE----------------2 EXPIRATION------0000.000
SMSDATA
STORAGECLASS -----SCTEST MANAGEMENTCLASS---MCTEST
DATACLASS ------VSAMKSEX LBACKUP ---0000.000.0000
CA-RECLAIM---------(YES)
EATTR-------------(NULL)
BWO STATUS------00000000 BWO TIMESTAMP---00000 00:00:00.0
BWO---------------(NULL)
RLSDATA
LOG ----------------(NULL) RECOVERY REQUIRED --(NO) FRLOG --------
----(NULL)
VSAM QUIESCED -------(NO) RLS IN USE ---------(NO) LOGREPLICATE--
-----------(NO)
LOGSTREAMID-----------------------------(NULL)
RECOVERY TIMESTAMP LOCAL-----X'0000000000000000'
RECOVERY TIMESTAMP GMT-------X'0000000000000000'
PROTECTION-PSWD-----(NULL) RACF----------------(NO)
ASSOCIATIONS
DATA-----TEST.SYLFA4V.VTST.MEMBER.CLUSTER.DATA
INDEX----TEST.SYLFA4V.VTST.MEMBER.CLUSTER.INDEX
DATA ------- TEST.SYLFA4V.VTST.MEMBER.CLUSTER.DATA
IN-CAT --- CATALOG.VNRO814
HISTORY
DATASET-OWNER-----(NULL) CREATION--------2018.005
RELEASE----------------2 EXPIRATION------0000.000
ACCOUNT-INFO-----------------------------------(NULL)
PROTECTION-PSWD-----(NULL) RACF----------------(NO)
ASSOCIATIONS
CLUSTER--TEST.SYLFA4V.VTST.MEMBER.CLUSTER
ATTRIBUTES
KEYLEN----------------32 AVGLRECL-------------170 BUFSPACE------
-----57344 CISIZE-------------26624
RKP--------------------0 MAXLRECL-----------11950 EXCPEXIT------
----(NULL) CI/CA-----------------30
STRIPE-COUNT-----------1
SHROPTNS(2,3) SPEED UNIQUE NOERASE INDEXED
NOWRITECHK UNORDERED REUSE
NONSPANNED EXTENDED EXT-ADDR
STATISTICS
REC-TOTAL-------32455840 SPLITS-CI--------------0 EXCPS---------
----255142
REC-DELETED------------0 SPLITS-CA--------------0 EXTENTS-------
--------95
REC-INSERTED-----------0 FREESPACE-%CI----------5 SYSTEM-TIMESTAMP:
REC-UPDATED------------0 FREESPACE-%CA----------5 X'D3B1D48
159E35E41'
REC-RETRIEVED--128852464 FREESPC--------971616256
ALLOCATION
SPACE-TYPE------CYLINDER HI-A-RBA-----29142097920
SPACE-PRI------------800 HI-U-RBA-----29142097920
SPACE-SEC------------300
VOLUME
VOLSER------------E30341 PHYREC-SIZE--------26624 HI-A-RBA------
3035136000 EXTENT-NUMBER---------11
DEVTYPE------X'3010200F' PHYRECS/TRK------------2 HI-U-RBA------
3035136000 EXTENT-TYPE--------X'40'
VOLFLAG------------PRIME TRACKS/CA-------------15
EXTENTS:
LOW-CCHH-----X'13190000' LOW-RBA----------------0 TRACKS--------
-----12000
HIGH-CCHH----X'1638000E' HIGH-RBA-------638975999
LOW-CCHH-----X'0E380000' LOW-RBA--------638976000 TRACKS--------
------9000
HIGH-CCHH----X'108F000E' HIGH-RBA------1118207999
LOW-CCHH-----X'11AF0000' LOW-RBA-------1118208000 TRACKS--------
------4500
HIGH-CCHH----X'12DA000E' HIGH-RBA------1357823999

LOW-CCHH-----X'171F0000' LOW-RBA-------1357824000 TRACKS--------
-----13500
HIGH-CCHH----X'1AA2000E' HIGH-RBA------2076671999
LOW-CCHH-----X'25110000' LOW-RBA-------2076672000 TRACKS--------
------4500
HIGH-CCHH----X'263C000E' HIGH-RBA------2316287999
LOW-CCHH-----X'1AA30000' LOW-RBA-------2316288000 TRACKS--------
------4050
HIGH-CCHH----X'1BB0000E' HIGH-RBA------2531942399
LOW-CCHH-----X'07690000' LOW-RBA-------2531942400 TRACKS--------
-------450
HIGH-CCHH----X'0786000E' HIGH-RBA------2555903999
LOW-CCHH-----X'263D0000' LOW-RBA-------2555904000 TRACKS--------
------3420
HIGH-CCHH----X'2720000E' HIGH-RBA------2738012159
LOW-CCHH-----X'07870000' LOW-RBA-------2738012160 TRACKS--------
------1080
HIGH-CCHH----X'07CE000E' HIGH-RBA------2795519999
LOW-CCHH-----X'16390000' LOW-RBA-------2795520000 TRACKS--------
------2940
HIGH-CCHH----X'16FC000E' HIGH-RBA------2952069119
LOW-CCHH-----X'10900000' LOW-RBA-------2952069120 TRACKS--------
------1560
HIGH-CCHH----X'10F7000E' HIGH-RBA------3035135999
VOLUME
VOLSER------------E30193 PHYREC-SIZE--------26624 HI-A-RBA------
5191680000 EXTENT-NUMBER----------7
DEVTYPE------X'3010200F' PHYRECS/TRK------------2 HI-U-RBA------
5191680000 EXTENT-TYPE--------X'40'
VOLFLAG------------PRIME TRACKS/CA-------------15
EXTENTS:
LOW-CCHH-----X'0A180000' LOW-RBA-------3035136000 TRACKS--------
-----18000
HIGH-CCHH----X'0EC7000E' HIGH-RBA------3993599999
LOW-CCHH-----X'19CE0000' LOW-RBA-------3993600000 TRACKS--------
-----13500
HIGH-CCHH----X'1D51000E' HIGH-RBA------4712447999
LOW-CCHH-----X'12520000' LOW-RBA-------4712448000 TRACKS--------
------2685
HIGH-CCHH----X'1304000E' HIGH-RBA------4855418879
LOW-CCHH-----X'037C0000' LOW-RBA-------4855418880 TRACKS--------
------1815
HIGH-CCHH----X'03F4000E' HIGH-RBA------4952063999
LOW-CCHH-----X'1D520000' LOW-RBA-------4952064000 TRACKS--------
------2505
HIGH-CCHH----X'1DF8000E' HIGH-RBA------5085450239

INDEX ------ TEST.SYLFA4V.VTST.MEMBER.CLUSTER.INDEX
IN-CAT --- CATALOG.VNRO814
HISTORY
DATASET-OWNER-----(NULL) CREATION--------2018.005
RELEASE----------------2 EXPIRATION------0000.000
PROTECTION-PSWD-----(NULL) RACF----------------(NO)
ASSOCIATIONS
CLUSTER--TEST.SYLFA4V.VTST.MEMBER.CLUSTER
ATTRIBUTES
KEYLEN----------------32 AVGLRECL---------------0 BUFSPACE------
---------0 CISIZE--------------4096
RKP--------------------0 MAXLRECL------------4089 EXCPEXIT------
----(NULL) CI/CA-----------------12
SHROPTNS(2,3) RECOVERY UNIQUE NOERASE NOWRITECHK
UNORDERED REUSE EXTENDED
EXT-ADDR
STATISTICS
REC-TOTAL----------36622 SPLITS-CI--------------0 EXCPS---------
----331595 INDEX:
REC-DELETED------------0 SPLITS-CA--------------0 EXTENTS-------
---------3 LEVELS-----------------3
REC-INSERTED-----------0 FREESPACE-%CI----------0 SYSTEM-TIMESTA
MP: ENTRIES/SECT-----------5
REC-UPDATED------------0 FREESPACE-%CA----------0 X'D3B1D48
159E35E41' SEQ-SET-RBA----------------0
REC-RETRIEVED----------0 FREESPC----------1138688
HI-LEVEL-RBA----------995328
ALLOCATION
SPACE-TYPE------CYLINDER HI-A-RBA-------151142400
SPACE-PRI-------------55 HI-U-RBA-------150003712
SPACE-SEC-------------50
VOLUME
VOLSER------------E30341 PHYREC-SIZE---------4096 HI-A-RBA------
-151142400 EXTENT-NUMBER----------3 DEVTYPE------X'3010200F' PHYRECS/TRK-----------12 HI-U-RBA------
-150003712 EXTENT-TYPE--------X'40'
VOLFLAG------------PRIME TRACKS/CA--------------1
EXTENTS:
LOW-CCHH-----X'07320000' LOW-RBA----------------0 TRACKS--------
-------825
HIGH-CCHH----X'0768000E' HIGH-RBA--------40550399
LOW-CCHH-----X'07CF0000' LOW-RBA---------40550400 TRACKS--------
------1500
HIGH-CCHH----X'0832000E' HIGH-RBA-------114278399
LOW-CCHH-----X'10F80000' LOW-RBA--------114278400 TRACKS--------
-------750
HIGH-CCHH----X'1129000E' HIGH-RBA-------151142399
VOLUME
VOLSER-----------------* PHYREC-SIZE------------0 HI-A-RBA------
---------0 EXTENT-NUMBER----------0
DEVTYPE------X'3010200F' PHYRECS/TRK------------0 HI-U-RBA------
---------0 EXTENT-TYPE--------X'FF'
VOLFLAG--------CANDIDATE TRACKS/CA--------------0
VOLUME
VOLSER-----------------* PHYREC-SIZE------------0 HI-A-RBA------
---------0 EXTENT-NUMBER----------0
DEVTYPE------X'3010200F' PHYRECS/TRK------------0 HI-U-RBA------

===============================================================================================================

DATASET-OWNER-----(NULL) CREATION--------2018.005
RELEASE----------------2 EXPIRATION------0000.000
SMSDATA
STORAGECLASS -----SCTEST MANAGEMENTCLASS---MCTEST
DATACLASS ------VSAMKSEX LBACKUP ---0000.000.0000
CA-RECLAIM---------(YES)
EATTR-------------(NULL)
BWO STATUS------00000000 BWO TIMESTAMP---00000 00:00:00.0
BWO---------------(NULL)
RLSDATA
LOG ----------------(NULL) RECOVERY REQUIRED --(NO) FRLOG --------
----(NULL)
VSAM QUIESCED -------(NO) RLS IN USE ---------(NO) LOGREPLICATE--
-----------(NO)
LOGSTREAMID-----------------------------(NULL)
RECOVERY TIMESTAMP LOCAL-----X'0000000000000000'
RECOVERY TIMESTAMP GMT-------X'0000000000000000'
PROTECTION-PSWD-----(NULL) RACF----------------(NO)
ASSOCIATIONS
DATA-----TEST.SYLFA4V.VTST.MEM.PREVDAY.CLUSTER.DATA
INDEX----TEST.SYLFA4V.VTST.MEM.PREVDAY.CLUSTER.INDEX
DATA ------- TEST.SYLFA4V.VTST.MEM.PREVDAY.CLUSTER.DATA
IN-CAT --- CATALOG.VNRO814
HISTORY
DATASET-OWNER-----(NULL) CREATION--------2018.005
RELEASE----------------2 EXPIRATION------0000.000
ACCOUNT-INFO-----------------------------------(NULL)
PROTECTION-PSWD-----(NULL) RACF----------------(NO)
ASSOCIATIONS
CLUSTER--TEST.SYLFA4V.VTST.MEM.PREVDAY.CLUSTER
ATTRIBUTES
KEYLEN----------------32 AVGLRECL-------------170 BUFSPACE------
-----57344 CISIZE-------------26624
RKP--------------------0 MAXLRECL-----------11950 EXCPEXIT------
----(NULL) CI/CA-----------------30
STRIPE-COUNT-----------1
SHROPTNS(2,3) SPEED UNIQUE NOERASE INDEXED
NOWRITECHK UNORDERED REUSE
NONSPANNED EXTENDED EXT-ADDR
STATISTICS
REC-TOTAL-------32448318 SPLITS-CI--------------0 EXCPS---------
----219453
REC-DELETED------------0 SPLITS-CA--------------0 EXTENTS-------
--------53
REC-INSERTED-----------0 FREESPACE-%CI----------5 SYSTEM-TIMESTAMP:
REC-UPDATED------------0 FREESPACE-%CA----------5 X'D3B1D49
775DAFA41'
REC-RETRIEVED---96589377 FREESPC--------971696128
ALLOCATION
SPACE-TYPE------CYLINDER HI-A-RBA-----29137305600
SPACE-PRI------------800 HI-U-RBA-----29137305600
SPACE-SEC------------500
VOLUME
VOLSER------------E30314 PHYREC-SIZE--------26624 HI-A-RBA------
3833856000 EXTENT-NUMBER----------3
DEVTYPE------X'3010200F' PHYRECS/TRK------------2 HI-U-RBA------
3833856000 EXTENT-TYPE--------X'40'
VOLFLAG------------PRIME TRACKS/CA-------------15
EXTENTS:
LOW-CCHH-----X'16590000' LOW-RBA----------------0 TRACKS--------
-----64440
HIGH-CCHH----X'2720000E' HIGH-RBA------3431301119
LOW-CCHH-----X'10680000' LOW-RBA-------3431301120 TRACKS--------
------6630
HIGH-CCHH----X'1221000E' HIGH-RBA------3784335359
LOW-CCHH-----X'0F9D0000' LOW-RBA-------3784335360 TRACKS--------
-------930
HIGH-CCHH----X'0FDA000E' HIGH-RBA------3833855999
VOLUME
VOLSER------------E30310 PHYREC-SIZE--------26624 HI-A-RBA------
5830656000 EXTENT-NUMBER----------3
DEVTYPE------X'3010200F' PHYRECS/TRK------------2 HI-U-RBA------
5830656000 EXTENT-TYPE--------X'40'
VOLFLAG------------PRIME TRACKS/CA-------------15
EXTENTS:
LOW-CCHH-----X'1F4B0000' LOW-RBA-------3833856000 TRACKS--------
-----30000
HIGH-CCHH----X'271A000E' HIGH-RBA------5431295999
LOW-CCHH-----X'1C3D0000' LOW-RBA-------5431296000 TRACKS--------
------5730
HIGH-CCHH----X'1DBA000E' HIGH-RBA------5736407039
LOW-CCHH-----X'19230000' LOW-RBA-------5736407040 TRACKS--------
------1770
HIGH-CCHH----X'1998000E' HIGH-RBA------5830655999
VOLUME
VOLSER------------E30335 PHYREC-SIZE--------26624 HI-A-RBA------
9025536000 EXTENT-NUMBER----------1
DEVTYPE------X'3010200F' PHYRECS/TRK------------2 HI-U-RBA------
9025536000 EXTENT-TYPE--------X'40'
VOLFLAG------------PRIME TRACKS/CA-------------15
EXTENTS:
LOW-CCHH-----X'0C940000' LOW-RBA-------5830656000 TRACKS--------
-----60000
HIGH-CCHH----X'1C33000E' HIGH-RBA------9025535999
VOLUME
VOLSER------------E30072 PHYREC-SIZE--------26624 HI-A-RBA-----1
3019136000 EXTENT-NUMBER----------8
DEVTYPE------X'3010200F' PHYRECS/TRK------------2 HI-U-RBA-----1
3019136000 EXTENT-TYPE--------X'40'
VOLFLAG------------PRIME TRACKS/CA-------------15
EXTENTS:
LOW-CCHH-----X'01DB0000' LOW-RBA-------9025536000 TRACKS--------
------7500
HIGH-CCHH----X'03CE000E' HIGH-RBA------9424895999
LOW-CCHH-----X'0D060000' LOW-RBA-------9424896000 TRACKS--------
-----15000
HIGH-CCHH----X'10ED000E' HIGH-RBA-----10223615999
LOW-CCHH-----X'12AE0000' LOW-RBA------10223616000 TRACKS--------
------7500
HIGH-CCHH----X'14A1000E' HIGH-RBA-----10622975999
LOW-CCHH-----X'1E450000' LOW-RBA------10622976000 TRACKS--------
-----30000
HIGH-CCHH----X'2614000E' HIGH-RBA-----12220415999
LOW-CCHH-----X'03CF0000' LOW-RBA------12220416000 TRACKS--------
------4980
HIGH-CCHH----X'051A000E' HIGH-RBA-----12485591039
LOW-CCHH-----X'10EE0000' LOW-RBA------12485591040 TRACKS--------
------2520
HIGH-CCHH----X'1195000E' HIGH-RBA-----12619775999
LOW-CCHH-----X'26150000' LOW-RBA------12619776000 TRACKS--------
------4020
HIGH-CCHH----X'2720000E' HIGH-RBA-----12833832959
LOW-CCHH-----X'14A20000' LOW-RBA------12833832960 TRACKS--------
------3480
HIGH-CCHH----X'1589000E' HIGH-RBA-----13019135999
VOLUME
VOLSER------------E30311 PHYREC-SIZE--------26624 HI-A-RBA-----1
5015936000 EXTENT-NUMBER----------5
DEVTYPE------X'3010200F' PHYRECS/TRK------------2 HI-U-RBA-----1
5015936000 EXTENT-TYPE--------X'40'
VOLFLAG------------PRIME TRACKS/CA-------------15
EXTENTS:
LOW-CCHH-----X'0C500000' LOW-RBA------13019136000 TRACKS--------
------7500
HIGH-CCHH----X'0E43000E' HIGH-RBA-----13418495999
LOW-CCHH-----X'13250000' LOW-RBA------13418496000 TRACKS--------
-----15000
VOLFLAG--------CANDIDATE TRACKS/CA--------------0
INDEX ------ TEST.SYLFA4V.VTST.MEM.PREVDAY.CLUSTER.INDEX
IN-CAT --- CATALOG.VNRO814
HISTORY
DATASET-OWNER-----(NULL) CREATION--------2018.005
RELEASE----------------2 EXPIRATION------0000.000
PROTECTION-PSWD-----(NULL) RACF----------------(NO)
ASSOCIATIONS
CLUSTER--TEST.SYLFA4V.VTST.MEM.PREVDAY.CLUSTER
ATTRIBUTES
KEYLEN----------------32 AVGLRECL---------------0 BUFSPACE------
---------0 CISIZE--------------4096
RKP--------------------0 MAXLRECL------------4089 EXCPEXIT------
----(NULL) CI/CA-----------------12
SHROPTNS(2,3) RECOVERY UNIQUE NOERASE NOWRITECHK
UNORDERED REUSE EXTENDED
EXT-ADDR
STATISTICS
REC-TOTAL----------36616 SPLITS-CI--------------0 EXCPS---------
----295548 INDEX:
REC-DELETED------------0 SPLITS-CA--------------0 EXTENTS-------
---------3 LEVELS-----------------3
REC-INSERTED-----------0 FREESPACE-%CI----------0 SYSTEM-TIMESTA
MP: ENTRIES/SECT-----------5
REC-UPDATED------------0 FREESPACE-%CA----------0 X'D3B1D49
775DAFA41' SEQ-SET-RBA----------------0
REC-RETRIEVED----------0 FREESPC----------1163264
HI-LEVEL-RBA----------987136
ALLOCATION
SPACE-TYPE------CYLINDER HI-A-RBA-------151142400
SPACE-PRI-------------55 HI-U-RBA-------149979136
SPACE-SEC-------------50
VOLUME
VOLSER------------E30314 PHYREC-SIZE---------4096 HI-A-RBA------
-151142400 EXTENT-NUMBER----------3
DEVTYPE------X'3010200F' PHYRECS/TRK-----------12 HI-U-RBA------
-149979136 EXTENT-TYPE--------X'40'
VOLFLAG------------PRIME TRACKS/CA--------------1
EXTENTS:
LOW-CCHH-----X'0F660000' LOW-RBA----------------0 TRACKS--------
-------825
HIGH-CCHH----X'0F9C000E' HIGH-RBA--------40550399
LOW-CCHH-----X'0FDB0000' LOW-RBA---------40550400 TRACKS--------
------2100
HIGH-CCHH----X'1066000E' HIGH-RBA-------143769599
LOW-CCHH-----X'0EF00000' LOW-RBA--------143769600 TRACKS--------
-------150
HIGH-CCHH----X'0EF9000E' HIGH-RBA-------151142399
VOLUME
VOLSER-----------------* PHYREC-SIZE------------0 HI-A-RBA------
---------0 EXTENT-NUMBER----------0
DEVTYPE------X'3010200F' PHYRECS/TRK------------0 HI-U-RBA------
---------0 EXTENT-TYPE--------X'FF'
VOLFLAG--------CANDIDATE TRACKS/CA--------------0
VOLUME
VOLSER-----------------* PHYREC-SIZE------------0 HI-A-RBA------
---------0 EXTENT-NUMBER----------0
DEVTYPE------X'3010200F' PHYRECS/TRK------------0 HI-U-RBA------
---------0 EXTENT-TYPE--------X'FF'
VOLFLAG--------CANDIDATE TRACKS/CA--------------0
VOLUME
VOLSER-----------------* PHYREC-SIZE------------0 HI-A-RBA------
---------0 EXTENT-NUMBER----------0
DEVTYPE------X'3010200F' PHYRECS/TRK------------0 HI-U-RBA------
---------0 EXTENT-TYPE--------X'FF'
VOLFLAG--------CANDIDATE TRACKS/CA--------------0
VOLUME
VOLSER-----------------* PHYREC-SIZE------------0 HI-A-RBA------
---------0 EXTENT-NUMBER----------0
DEVTYPE------X'3010200F' PHYRECS/TRK------------0 HI-U-RBA------
---------0 EXTENT-TYPE--------X'FF'
VOLFLAG--------CANDIDATE TRACKS/CA--------------0
VOLUME
VOLSER-----------------* PHYREC-SIZE------------0 HI-A-RBA------
---------0 EXTENT-NUMBER----------0
DEVTYPE------X'3010200F' PHYRECS/TRK------------0 HI-U-RBA------
---------0 EXTENT-TYPE--------X'FF'
VOLFLAG--------CANDIDATE TRACKS/CA--------------0
VOLUME
VOLSER-----------------* PHYREC-SIZE------------0 HI-A-RBA------
---------0 EXTENT-NUMBER----------0
DEVTYPE------X'3010200F' PHYRECS/TRK------------0 HI-U-RBA------
---------0 EXTENT-TYPE--------X'FF'
VOLFLAG--------CANDIDATE TRACKS/CA--------------0
VOLUME
VOLSER-----------------* PHYREC-SIZE------------0 HI-A-RBA------
---------0 EXTENT-NUMBER----------0
DEVTYPE------X'3010200F' PHYRECS/TRK------------0 HI-U-RBA------
---------0 EXTENT-TYPE--------X'FF'
VOLFLAG--------CANDIDATE TRACKS/CA--------------0

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Arun Venkatratnam

2018-01-06 17:31:11 UTC

Permalink

1. Have you looked at adding STRNO on the VSAM definition or using RLS to provide the records to be instorage?

No, STRNO is not provided explicitly in the VSAM file definition.Strobe apparently shows the number of Strings is 48. RLS is also not used.

2. In your JCL for the job, have you coded BUFND, or BUFNI or BUFSP?

Yes BUFND, BUFNI were provided.

TEST.SYLFA4V.VTST.MEMBER.CLUSTER - BUFNI = 30, BUND=225. This file is accessed sequentially
TEST.SYLFA4V.VTST.MEMBER.PREVDAY.CLUSTER BUFNI = 120, BUFND = 240. This file is accessed skip-sequentially.

3. Could you provide the LISTC ENT(vsam-name-here) ALL for us to review?

Is the File KSDS, ESD, or Does it have INDEX/PATH/AIX?
These are VSAM KSDS files, no AIX/PATH.

4. What are your share options?

Share options are 2,3 . We also tried with 1,3

5. When was the Cobol program last compiled and lked?

The program was compiled on 03/31/2017

6. What version of Cobol was being used when it was last compiled and lked?

IBM ENTERPRISE COBOL FOR Z/OS 4.2.0

7. Has a review of the Cobol program logic been done to see if any performance can be gotten there?

Not yet, we are looking to improve performance of all programs that do VSAM reads (mainly the ones that do SKIP SEQUENTIAL reads)

8. What version of z/OS are you running on?

z/OS 02.01.00

9. Are there any other processes using this VSAM dataset when it is being accessed by this program? DB2, CICS, etc....

No, there are no other processes accessing the file.

10. Have you reviewed RMF Data to see if the DASD was performing well? Or if the program was preforming well

RMF Disk activity reports were not reviewed. But, we did a stand-alone test to see if it was sequential access or skip sequential reads was the cause of high CPU.

Resp: Reading through the entire file by sequential access took 90 CPU seconds while skip-sequential took nearly 230 CPU seconds.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Mike Schwab

2018-01-06 17:52:34 UTC

Permalink

On Sat, Jan 6, 2018 at 11:32 AM, Arun Venkatratnam
<***@cognizant.com> wrote:
<deleted>

Post by Arun Venkatratnam
Resp: Reading through the entire file by sequential access took 90 CPU seconds while skip-sequential took nearly 230 CPU seconds.

Ron hawkins

2018-01-07 10:46:09 UTC

Permalink

Mike and Arun,

I think the assumption here is that there is that there are no records in the second file (skip sequential) that are being accessed more than once by a foreign key from the first file. If that is the case, then I think it makes sense that a sorted copy of the first file using the same key as the 2nd file would make a single redundant file, and both can match/merged sequentially. I assume that the keys are different for each file, because if they are the same, then the program would have been written with both files processed sequentially in the first place.

However, let me cut to what I think the real problem may be, and that is that the 2nd file is over-buffered. The files have almost the same number of records, which means that one CI in the 2nd file probably has the only record accessed by foreign key from the first file. I suggest that you knock the BUFND value for the 2nd file down from 240 to just 2. At the moment you are reading 30 CI for every skip-seq access, (more with zHPF), and probably just reading the first CI. It's a wild ass guess that all that buffer management, not to mention the connect time, are at the root of the CPU time you are trying to eliminate. The BUFND reduction follows my original recommendation to understand the ratio of 2nd file records accessed per record in the first file. You have SAS - read the records and analyze it.

TEST.SYLFA4V.VTST.MEMBER.CLUSTER - BUFNI = 30, BUND=225. This file is accessed sequentially
TEST.SYLFA4V.VTST.MEMBER.PREVDAY.CLUSTER BUFNI = 120, BUFND = 240. This file is accessed skip-sequentially.

FWIW the idea of using LSR for skip sequential access should probably be nixed if no record in the 2nd file is accessed more than once. All you will do is walk the buffer pool and never get a buffer hit. The near equal number of records supports the no-hit premise.

Ron

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Mike Schwab
Sent: Saturday, January 6, 2018 9:54 AM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

On Sat, Jan 6, 2018 at 11:32 AM, Arun Venkatratnam <***@cognizant.com> wrote:
<deleted>

Post by Arun Venkatratnam
Resp: Reading through the entire file by sequential access took 90 CPU seconds while skip-sequential took nearly 230 CPU seconds.

Doing the skip sequential (read by key) requires accessing the index to find the next record. Just skipping through non-matching records
is faster when you are reading a large subset of the records. A sort
to match up the records to create an extract for further processing could be even faster.

--
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Arun Venkatratnam

2018-01-07 18:20:17 UTC

Permalink

Ron,

About 80% of the records qualify from the second file. I tried running the job with BUFND of 2 for the second file. It increased the EXCPs significantly (from strobe report) on the data records for the 2nd file and the CPU time went over the other runs and the elapsed time also increased. The job runs better with higher values of BUFND for the second file.

You had mentioned that the job reads 30 CI for every skip-seq access. Could you please tell how did you arrive at that number. LISTCAT shows that for a CISZ of 26KB, there are 30 CI in 1 CA.

Thanks
Arun

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Gerhard Adam

2018-01-07 19:02:26 UTC

Permalink

To me it still looks like you don't have enough index buffers. My calculation suggests you need at least 137 buffers to keep the index set in memory.

Sent from my iPhone

Post by Arun Venkatratnam
Ron,
About 80% of the records qualify from the second file. I tried running the job with BUFND of 2 for the second file. It increased the EXCPs significantly (from strobe report) on the data records for the 2nd file and the CPU time went over the other runs and the elapsed time also increased. The job runs better with higher values of BUFND for the second file.
You had mentioned that the job reads 30 CI for every skip-seq access. Could you please tell how did you arrive at that number. LISTCAT shows that for a CISZ of 26KB, there are 30 CI in 1 CA.
Thanks
Arun
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

Wayne Bickerdike

2018-01-07 21:17:03 UTC

Permalink

Quote
Apologies. I missed to mention that depending on a record that matches in
the 2nd file, there are some random reads that refer back to a previous
record as per the business logic. Hence, the second file has to be VSAM. We
also see a similar behavior with few other programs that do skip sequential
reads on other VSAM files.
Unquote

So do the two file match and utilise the VSAM (second file) for random read
where it's required.

You won't tune your way out of this.

Post by Gerhard Adam
To me it still looks like you don't have enough index buffers. My
calculation suggests you need at least 137 buffers to keep the index set in
memory.
Sent from my iPhone

On Jan 7, 2018, at 10:21 AM, Arun Venkatratnam <
Ron,
About 80% of the records qualify from the second file. I tried running

the job with BUFND of 2 for the second file. It increased the EXCPs
significantly (from strobe report) on the data records for the 2nd file and
the CPU time went over the other runs and the elapsed time also increased.
The job runs better with higher values of BUFND for the second file.

You had mentioned that the job reads 30 CI for every skip-seq access.

Could you please tell how did you arrive at that number. LISTCAT shows that
for a CISZ of 26KB, there are 30 CI in 1 CA.

Thanks
Arun
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

Chris Hoelscher

2018-01-08 04:56:32 UTC

Permalink

How 'bout .....

Flagging the matches that currently require the random read - and instead pass those records thru the 2nd file - both in this random key sequence - that would eliminate all random reads - yes another sort or 2 and another match pass - but in the past I have found the savings of replacing random reads with sequential access is substantial

Chris Hoelscher
Technology Architect, Database Infrastructure Services
Technology Solution Services

123 East Main Street
Louisville, KY 40202
Humana.com
(502) 476-2538 or 407-7266

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Wayne Bickerdike
Sent: Sunday, January 7, 2018 4:18 PM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

Quote
Apologies. I missed to mention that depending on a record that matches in the 2nd file, there are some random reads that refer back to a previous record as per the business logic. Hence, the second file has to be VSAM. We also see a similar behavior with few other programs that do skip sequential reads on other VSAM files.
Unquote

So do the two file match and utilise the VSAM (second file) for random read where it's required.

You won't tune your way out of this.

Post by Gerhard Adam
To me it still looks like you don't have enough index buffers. My
calculation suggests you need at least 137 buffers to keep the index
set in memory.
Sent from my iPhone

On Jan 7, 2018, at 10:21 AM, Arun Venkatratnam <
Ron,
About 80% of the records qualify from the second file. I tried
running

the job with BUFND of 2 for the second file. It increased the EXCPs
significantly (from strobe report) on the data records for the 2nd
file and the CPU time went over the other runs and the elapsed time also increased.
The job runs better with higher values of BUFND for the second file.

You had mentioned that the job reads 30 CI for every skip-seq access.

Could you please tell how did you arrive at that number. LISTCAT shows
that for a CISZ of 26KB, there are 30 CI in 1 CA.

Thanks
Arun
--------------------------------------------------------------------
-- For IBM-MAIN subscribe / signoff / archive access instructions,

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send

--
Wayne V. Bickerdike

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

The information transmitted is intended only for the person or entity to which it is addressed
and may contain CONFIDENTIAL material. If you receive this material/information in error,
please contact the sender and delete or destroy the material/information.

Humana Inc. and its subsidiaries comply with applicable Federal civil rights laws and
do not discriminate on the basis of race, color, national origin, age, disability or
sex. Humana Inc. and its subsidiaries do not exclude people or treat them differently
because of race, color, national origin, age, disability or sex.

English: ATTENTION: If you do not speak English, language assistance services, free
of charge, are available to you. Call 1‐877‐320‐1235 (TTY: 711).

Español (Spanish): ATENCIÓN: Si habla español, tiene a su disposición servicios
gratuitos de asistencia lingüística. Llame al 1‐877‐320‐1235 (TTY: 711).

繁體中文(Chinese)：注意：如果您使用繁體中文，您可以免費獲得語言援助
服務。請致電 1‐877‐320‐1235 (TTY: 711)。

Kreyòl Ayisyen (Haitian Creole): ATANSION: Si w pale Kreyòl Ayisyen, gen sèvis èd
pou lang ki disponib gratis pou ou. Rele 1‐877‐320‐1235 (TTY: 711).

Polski (Polish): UWAGA: Jeżeli mówisz po polsku, możesz skorzystać z bezpłatnej
pomocy językowej. Zadzwoń pod numer 1‐877‐320‐1235 (TTY: 711).

한국어 (Korean): 주의: 한국어를 사용하시는 경우, 언어 지원 서비스를 무료로
이용하실 수 있습니다. 1‐877‐320‐1235 (TTY: 711)번으로 전화해 주십시오.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Ron hawkins

2018-01-08 14:32:08 UTC

Permalink

Arun,

OK, looking at the full strobe report, if I read it correctly, converting to sequential processing may not be the most efficient way to process this. Both files have 32.4M records, but you only touch 4.9M records in each file when the program executes. You are touching around 15% of the total records in each file so a full sequential read of each may cost more CPU time., especially if you include the time to build redundant files.

The distribution of touched cylinders in both files is quite random, with IEHEYEBALL telling me that the distribution and clustering of touched cylinders are similar in both files. The clustered activity supports your finding that there is some efficiency from more data chaining for the skip sequential IO.

I'm not sure what your objectives are for this. You have a program that processes 8 million records using 2 minutes of CPU time in a three minute period and doing 9.4K IO to each of the index and data components of both files. Strobe does not report the CPU model in your report, so I don't know if this is on z196-401, a z14-708, or something in between, but if you want an omelet, you have to crack a few eggs. The CPU time is not SRB time, which corresponds to the relatively low IO count.

Strobe support can tell you why IGZEQBL mentions QSAM in the description, but it does not mean you are spending time in QSAM. My wife's laptop is an Apple, but I can't eat it - it's a label. I think that the time in IQZEQBL and IGZCPCO are simply the cost of reading 8 million records from the buffers and not related to the actual IO operations. You need to engage a Strobe expert or Compuware to dig deeper into this.

Ron

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Chris Hoelscher
Sent: Sunday, January 7, 2018 8:57 PM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

How 'bout .....

Flagging the matches that currently require the random read - and instead pass those records thru the 2nd file - both in this random key sequence - that would eliminate all random reads - yes another sort or 2 and another match pass - but in the past I have found the savings of replacing random reads with sequential access is substantial

Chris Hoelscher
Technology Architect, Database Infrastructure Services Technology Solution Services

123 East Main Street
Louisville, KY 40202
Humana.com
(502) 476-2538 or 407-7266

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Wayne Bickerdike
Sent: Sunday, January 7, 2018 4:18 PM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

Quote
Apologies. I missed to mention that depending on a record that matches in the 2nd file, there are some random reads that refer back to a previous record as per the business logic. Hence, the second file has to be VSAM. We also see a similar behavior with few other programs that do skip sequential reads on other VSAM files.
Unquote

So do the two file match and utilise the VSAM (second file) for random read where it's required.

You won't tune your way out of this.

Post by Gerhard Adam
To me it still looks like you don't have enough index buffers. My
calculation suggests you need at least 137 buffers to keep the index
set in memory.
Sent from my iPhone

On Jan 7, 2018, at 10:21 AM, Arun Venkatratnam <
Ron,
About 80% of the records qualify from the second file. I tried
running

the job with BUFND of 2 for the second file. It increased the EXCPs
significantly (from strobe report) on the data records for the 2nd
file and the CPU time went over the other runs and the elapsed time also increased.
The job runs better with higher values of BUFND for the second file.

You had mentioned that the job reads 30 CI for every skip-seq access.

Could you please tell how did you arrive at that number. LISTCAT shows
that for a CISZ of 26KB, there are 30 CI in 1 CA.

Thanks
Arun
--------------------------------------------------------------------
-- For IBM-MAIN subscribe / signoff / archive access instructions,

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send

--
Wayne V. Bickerdike

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

The information transmitted is intended only for the person or entity to which it is addressed and may contain CONFIDENTIAL material. If you receive this material/information in error, please contact the sender and delete or destroy the material/information.

Humana Inc. and its subsidiaries comply with applicable Federal civil rights laws and do not discriminate on the basis of race, color, national origin, age, disability or sex. Humana Inc. and its subsidiaries do not exclude people or treat them differently because of race, color, national origin, age, disability or sex.

English: ATTENTION: If you do not speak English, language assistance services, free of charge, are available to you. Call 1‐877‐320‐1235 (TTY: 711).

Español (Spanish): ATENCIÓN: Si habla español, tiene a su disposición servicios gratuitos de asistencia lingüística. Llame al 1‐877‐320‐1235 (TTY: 711).

繁體中文(Chinese)：注意：如果您使用繁體中文，您可以免費獲得語言援助
服務。請致電 1‐877‐320‐1235 (TTY: 711)。

Kreyòl Ayisyen (Haitian Creole): ATANSION: Si w pale Kreyòl Ayisyen, gen sèvis èd pou lang ki disponib gratis pou ou. Rele 1‐877‐320‐1235 (TTY: 711).

Polski (Polish): UWAGA: Jeżeli mówisz po polsku, możesz skorzystać z bezpłatnej pomocy językowej. Zadzwoń pod numer 1‐877‐320‐1235 (TTY: 711).

한국어 (Korean): 주의: 한국어를 사용하시는 경우, 언어 지원 서비스를 무료로 이용하실 수 있습니다. 1‐877‐320‐1235 (TTY: 711)번으로 전화해 주십시오.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Lizette Koehler

2018-01-08 14:34:55 UTC

Permalink

I was wondering if this process might be more efficient if it was handled by a Data Base, IMS, DB2, etc...

I know when programs are developed the data is usually small and as it grows the performance gets affected by increased data it needs to handle.

Would this be one of those cases where the process is not great with VSAM but might be helped with a review with a Database in mind?

Lizette

-----Original Message-----
Behalf Of Wayne Bickerdike
Sent: Sunday, January 07, 2018 2:18 PM
Subject: Re: VSAM Performance - CPU reduction
Quote
Apologies. I missed to mention that depending on a record that matches in the
2nd file, there are some random reads that refer back to a previous record as
per the business logic. Hence, the second file has to be VSAM. We also see a
similar behavior with few other programs that do skip sequential reads on
other VSAM files.
Unquote
So do the two file match and utilise the VSAM (second file) for random read
where it's required.
You won't tune your way out of this.

Post by Gerhard Adam
To me it still looks like you don't have enough index buffers. My
calculation suggests you need at least 137 buffers to keep the index
set in memory.
Sent from my iPhone

On Jan 7, 2018, at 10:21 AM, Arun Venkatratnam <
Ron,
About 80% of the records qualify from the second file. I tried
running

the job with BUFND of 2 for the second file. It increased the EXCPs
significantly (from strobe report) on the data records for the 2nd
file and the CPU time went over the other runs and the elapsed time also

increased.

Post by Gerhard Adam
The job runs better with higher values of BUFND for the second file.

You had mentioned that the job reads 30 CI for every skip-seq access.

Could you please tell how did you arrive at that number. LISTCAT shows
that for a CISZ of 26KB, there are 30 CI in 1 CA.

Thanks
Arun

Ron hawkins

2018-01-08 00:23:45 UTC

Permalink

Gerhard,

He has BUFNI=120, so I don't think the occasional misses on the 2nd level
index set are a root cause.

It would show up in strobe as high IO to the index rather than data
component, or he has, I hope, looked at the type 64 and 42 subtype 6
records.

Ron

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On
Behalf Of Gerhard Adam
Sent: Sunday, January 7, 2018 11:04 AM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

To me it still looks like you don't have enough index buffers. My
calculation suggests you need at least 137 buffers to keep the index set in
memory.

Sent from my iPhone

On Jan 7, 2018, at 10:21 AM, Arun Venkatratnam
Ron,
About 80% of the records qualify from the second file. I tried running

You had mentioned that the job reads 30 CI for every skip-seq access.

Could you please tell how did you arrive at that number. LISTCAT shows that
for a CISZ of 26KB, there are 30 CI in 1 CA.

Thanks
Arun
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email
to ***@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Ron hawkins

2018-01-08 00:20:09 UTC

Permalink

Arun,

Would you share the whole strobe report?

You can send it to me directly as an attachment.

Standard VSAM and FICON does not chain across cylinder boundaries. 1 cyl = 1 CA, and with CISZ 26K there are 30 CI in a CA/cyl. Ipso facto you read 30 CI per SSCH. This increases with zHPF.

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Arun Venkatratnam
Sent: Sunday, January 7, 2018 10:22 AM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

Ron,

About 80% of the records qualify from the second file. I tried running the job with BUFND of 2 for the second file. It increased the EXCPs significantly (from strobe report) on the data records for the 2nd file and the CPU time went over the other runs and the elapsed time also increased. The job runs better with higher values of BUFND for the second file.

You had mentioned that the job reads 30 CI for every skip-seq access. Could you please tell how did you arrive at that number. LISTCAT shows that for a CISZ of 26KB, there are 30 CI in 1 CA.

Thanks
Arun

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Ron hawkins

2018-01-08 00:33:18 UTC

Permalink

So what is the max number of records processed per skip sequential access? However many CI that is, you should set BUFND to be that times 2.

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Arun Venkatratnam
Sent: Sunday, January 7, 2018 10:22 AM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

Ron,

About 80% of the records qualify from the second file. I tried running the job with BUFND of 2 for the second file. It increased the EXCPs significantly (from strobe report) on the data records for the 2nd file and the CPU time went over the other runs and the elapsed time also increased. The job runs better with higher values of BUFND for the second file.

You had mentioned that the job reads 30 CI for every skip-seq access. Could you please tell how did you arrive at that number. LISTCAT shows that for a CISZ of 26KB, there are 30 CI in 1 CA.

Thanks
Arun

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Arun Venkatratnam

2018-01-08 03:43:27 UTC

Permalink

Thanks Ron for the CI calculation, I have sent you the strobe report.

Thanks
Arun

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Arun Venkatratnam

2018-01-08 15:09:28 UTC

Permalink

Ron,

The job was not profiled for its entire run. All 32M records are read from the 1st file and 80% will qualify from the 2nd file. The job was profiled to capture 4000 samples in an elapsed time of 3 minutes. As I had mentioned in my initial post, the job takes nearly 7 minutes of CPU. The profile that this strobe report had captured is 1 CPU minute processing 4 M records. I will capture a strobe report for a full run of the job this evening.

The CPU model is a Z13S - S04

Thanks
Arun

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Arun Venkatratnam

2018-01-09 04:40:44 UTC

Permalink

Hi Ron and all,

I have captured the strobe report for the full run of the job. I will try to send it to later. In addition to strobe, i snapped the below statistics of the job from Mainview. The report shows that the EXCP and the SSCH counts are almost equal. I am of the understanding that if SSCH count can be reduced it can help in CPU reduction. Could you please provide some direction on to reduce the SSCH count.

FYI,I also tried with data CISZ of 18K, 30K and 32K. 18K and 26K look to show almost similar performance of taking 6.5 CPU minutes for the test files with 32M records. 26K CISZ showed the least EXCP counts amongst all..

CURR WIN ===> 1 ALT WIN ===>

W1 =JDELAYZ==JINFO====B33N=====*========08JAN2018==15:19:20====MVMVS====D====1

Timeframe... Interval Owner....... SYLFA4V 0...
Jobname..... TSTJEMEM Avg Frames.. 5120 Workflow... 7.46
Step/Proc... STEP1 Avg Cframes. 5120 Total Use.. 7.46
Job Step Mon ******** Avg Eframes. 0 Using Proc 2.99
JES Number.. JOB82957 Cframes held 5125 Using Dev. 4.48
Type........ BAT Eframes held 0 Total Dly.. 92.54
JobClass.... X Fixed frames 259 Processor. 92.54
JES Queue Tm 00:00:01 Fixed <16M.. 0 Device.... 0.00
%CPU Util... 7.4 Fixed 16-2G. 38 Storage... 0.00
Total CPU Tm 00:04:06 Dmd Page/Sec 0 Enqueue... 0.00
Totl ECPU Tm 00:04:06 Swp Page/Sec 0 SRM....... 0.00
Tot EXCP Cnt 106547 Avg UIC..... N/A Msg....... 0.00
Terminal ID. Avg Wkg Set. 0.0 XCF....... 0.00
ASID........ 364 SU/Sec...... 5027 JES....... 0.00
SrvClass.... BATMD EXCP/Sec.... 55.8 HSM....... 0.00
Workload.... BATCH JobStart Dt. 08JAN2018 Idle....... 0.00
%Connected.. 6.3 JobStart Tm. 14:29:46 ECB/Other.. 0.00
Disp. Prty.. 206 Job Elpd Tm. 00:49:27 Sec Userid.
Job Status.. Active JobEnd Dt... N/A Sec Group..

Jobname SSCH I/O% DDname Volser Addr DISP Data Set Name
------- ------- ----- -------- ------ ---- ---- ----------------------------
TSTJEMEM 31975 29.2 PREVMEM E30181 C0BB SHR TEST.SYLFA4V.VTST.MEM1.PREVD
TSTJEMEM 31440 28.7 CURRMEM E30415 C584 SHR TEST.SYLFA4V.VTST.MEMBER1.CL
TSTJEMEM 6300 5.7 CURRMEM E30162 C313 SHR TEST.SYLFA4V.VTST.MEMBER1.CL
TSTJEMEM 6001 5.5 PREVMEM E30316 C557 SHR TEST.SYLFA4V.VTST.MEM1.PREVD
TSTJEMEM 4000 3.6 PREVMEM E30364 C523 SHR TEST.SYLFA4V.VTST.MEM1.PREVD
TSTJEMEM 3752 3.4 CURRMEM E30448 C46E SHR TEST.SYLFA4V.VTST.MEMBER1.CL
TSTJEMEM 3476 3.2 CURRMEM E30121 C235 SHR TEST.SYLFA4V.VTST.MEMBER1.CL
TSTJEMEM 3300 3.0 CURRMEM E30258 C136 SHR TEST.SYLFA4V.VTST.MEMBER1.CL
TSTJEMEM 3000 2.7 PREVMEM E30347 C502 SHR TEST.SYLFA4V.VTST.MEM1.PREVD
TSTJEMEM 3000 2.7 PREVMEM E30186 C019 SHR TEST.SYLFA4V.VTST.MEM1.PREVD
TSTJEMEM 2762 2.5 PREVMEM E30063 C1B8 SHR TEST.SYLFA4V.VTST.MEM1.PREVD
TSTJEMEM 2421 2.2 PREVMEM E30247 C11F SHR TEST.SYLFA4V.VTST.MEM1.PREVD
TSTJEMEM 2400 2.2 CURRMEM E30076 C208 SHR TEST.SYLFA4V.VTST.MEMBER1.CL
TSTJEMEM 2100 1.9 CURRMEM E30187 C01A SHR TEST.SYLFA4V.VTST.MEMBER1.CL
TSTJEMEM 1500 1.4 PREVMEM E30392 C5AA SHR TEST.SYLFA4V.VTST.MEM1.PREVD
TSTJEMEM 1500 1.4 CURRMEM E30198 C025 SHR TEST.SYLFA4V.VTST.MEMBER1.CL
TSTJEMEM 297 0.3 DVFILE E30403 C578 SHR TEST.SYLFA4V.VTST.GROUP.CLUS

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Tim Hare

2018-01-15 06:59:13 UTC

Permalink

Unless things have changed, a 26K CI size is better than 32K because two CIs fit on a track. If I'm understanding this application it:

1. reads everything from the first file
2. for each record in the first file, it attempts to do a random read to find a match in the second file
3. IF it finds a match in the second file, then it might need to read one or more _other_ records in the second file - positioned by some value in the record that matched?

Seems to me you want a 26K CI size and at last BUFND=60 on the first file, to maximize sequential read of that file. Going too large on BUFND might be a waste, since your processing of the records in each CI are limited by what you're doing in the second file.

You definitely want a large BUFNI= for the second file because you don't want to do I/O for the key values. I'm not sure how much a large BUFND buys you - it depends how the related records (the ones you are going to retrieve depending on program logic) are stored and how many of them you need to read. As someone else said, reading too many of them will cost you.

Are the related records read using some alternate key? If so you might want a third 'file' that also reads that file, with index buffers for that alternate key.

One other thing - are these updated a lot? If updates are infrequent, or if performance can stand it, reduce the free space in the VSAM files, so that you're not reading in a lot of "empty" space in the CIs.

These are just suggestions, of course, without know a lot about your data.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Ron hawkins

2018-01-16 14:50:42 UTC

Permalink

Arun,

Don't bother, mate. I kind of give up.

People can't help when you keep the kimono shut.

I recommend you engage someone that works with Strobe, or the vendor directly.

Ron

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Sankaranarayanan, Vignesh

2018-01-17 07:36:46 UTC

Permalink

Reading IBM L3's answer to this kind of question would be a 'delicious' experience ... if you have IBM s/w support that is.

Something to consider... optimising this to save CPU time may not mean much if this job isn't running during a peak hour for the CEC.
If it does run at a busy time, and if it costs a lot of CPU, you may also want to just consider moving the job to a quieter window.
Sure, job chains, etc., but if it is worth it...

Also, since this is a 'scale' problem, you could look in StackOverflow to see how folks have solved similar problems in distributed platforms.
Just don't search for 'mainframe', 'VSAM', etc.... look for solutions on how to read through a huge file, and how to process another file based on the first read through.
Some of the answers/posters in SO have seen the depths of scaling hell ;)

– Vignesh
Mainframe Infrastructure

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Ron hawkins
Sent: 16 January 2018 14:52
To: IBM-***@LISTSERV.UA.EDU
Subject: [EXTERNAL] Re: VSAM Performance - CPU reduction

Arun,

Don't bother, mate. I kind of give up.

People can't help when you keep the kimono shut.

I recommend you engage someone that works with Strobe, or the vendor directly.

Ron

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

MARKSANDSPENCER.COM
________________________________
Unless otherwise stated above:
Marks and Spencer plc
Registered Office:
Waterside House
35 North Wharf Road
London
W2 1NW

Registered No. 214436 in England and Wales.

Telephone (020) 7935 4422
Facsimile (020) 7487 2670

www.marksandspencer.com

Please note that electronic mail may be monitored.

This e-mail is confidential. If you received it by mistake, please let us know and then delete it from your system; you should not copy, disclose, or distribute its contents to anyone nor act in reliance on this e-mail, as this is prohibited and may be unlawful.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Ron hawkins

2018-01-16 14:54:52 UTC

Permalink

Tim,

Things changed a couple of decades ago.

VSAM physical block is not always the same as the CISZ.

32K CISZ does not waste any space on the track.

Ron

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Clark Morris

2018-01-16 16:59:19 UTC

Permalink

[Default] On 16 Jan 2018 06:54:52 -0800, in bit.listserv.ibm-main

Post by Ron hawkins
Tim,
Things changed a couple of decades ago.
VSAM physical block is not always the same as the CISZ.
32K CISZ does not waste any space on the track.

Where is this written up? Being retired, I missed this and didn't
realize this was true on the OS390 system I worked on in the 1990s.

Clark Morris

Post by Ron hawkins
Ron
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

Mike Schwab

2018-01-16 23:06:26 UTC

Permalink

https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.idad400/ciopt.htm
Physical blocks in a CI is multiple of 512 up to 4096 bytes.

On Tue, Jan 16, 2018 at 11:00 AM, Clark Morris

Post by Clark Morris
[Default] On 16 Jan 2018 06:54:52 -0800, in bit.listserv.ibm-main

Post by Ron hawkins
Tim,
Things changed a couple of decades ago.
VSAM physical block is not always the same as the CISZ.
32K CISZ does not waste any space on the track.

Where is this written up? Being retired, I missed this and didn't
realize this was true on the OS390 system I worked on in the 1990s.
Clark Morris

Post by Ron hawkins
Ron
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

Ron hawkins

2018-01-19 13:38:00 UTC

Permalink

Mike,

I don't see where the link mentions the 4KiB physical block limitation as 4KiB.

It does not match with the physical record size used in my sample listcat.

This item does not give the calculation method, but it shows how CI uses multiple PB.

https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.idad400/d4222.htm

Ron

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Mike Schwab
Sent: Tuesday, January 16, 2018 3:08 PM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.idad400/ciopt.htm
Physical blocks in a CI is multiple of 512 up to 4096 bytes.

On Tue, Jan 16, 2018 at 11:00 AM, Clark Morris

Post by Clark Morris
[Default] On 16 Jan 2018 06:54:52 -0800, in bit.listserv.ibm-main

Post by Ron hawkins
Tim,
Things changed a couple of decades ago.
VSAM physical block is not always the same as the CISZ.
32K CISZ does not waste any space on the track.

Where is this written up? Being retired, I missed this and didn't
realize this was true on the OS390 system I worked on in the 1990s.
Clark Morris

Post by Ron hawkins
Ron
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

--
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Ron hawkins

2018-01-19 13:29:54 UTC

Permalink

Clark,

I don't know where it is written up, but it has been this way as long as I
can remember, which is 5 minutes when I go to the shop, and a bit longer for
IDCAMS.

Following example look at the CISZ(32768) versus the physical record size
(16384). Physical record size is the block size used for each record on the
track, so each CI uses two blocks or records and can span a track boundary.

DATA ------- HAWKINS.KSDS.DATA

IN-CAT --- CATALOG.VHDSUSR

HISTORY

DATASET-OWNER-----(NULL) CREATION--------2018.019

RELEASE----------------2 EXPIRATION------0000.000

ACCOUNT-INFO-----------------------------------(NULL)

PROTECTION-PSWD-----(NULL) RACF----------------(NO)

ASSOCIATIONS

CLUSTER--HAWKINS.KSDS

ATTRIBUTES

KEYLEN----------------44 AVGLRECL-------------435
BUFSPACE----------530432 CISIZE-------------32768
RKP--------------------0 MAXLRECL------------2040
EXCPEXIT----------(NULL) CI/CA-----------------22
SHROPTNS(3,3) SPEED UNIQUE NOERASE INDEXED
NOWRITECHK UNORDERED NOREUSE
NONSPANNED

STATISTICS

REC-TOTAL--------------0 SPLITS-CI--------------0
EXCPS------------------0
REC-DELETED------------0 SPLITS-CA--------------0
EXTENTS----------------1
REC-INSERTED-----------0 FREESPACE-%CI----------0
SYSTEM-TIMESTAMP:
REC-UPDATED------------0 FREESPACE-%CA----------0
X'0000000000000000'
REC-RETRIEVED----------0 FREESPC-----------720896

ALLOCATION

SPACE-TYPE------CYLINDER HI-A-RBA----------720896

SPACE-PRI--------------1 HI-U-RBA---------------0

SPACE-SEC--------------1

VOLUME

VOLSER------------HDSUS5 PHYREC-SIZE--------16384
HI-A-RBA----------720896 EXTENT-NUMBER----------1
DEVTYPE------X'3010200F' PHYRECS/TRK------------3
HI-U-RBA---------------0 EXTENT-TYPE--------X'40'
VOLFLAG------------PRIME TRACKS/CA-------------15

EXTENTS:

LOW-CCHH-----X'00060000' LOW-RBA----------------0
TRACKS----------------15
HIGH-CCHH----X'0006000E' HIGH-RBA----------720895

Ron

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On
Behalf Of Clark Morris
Sent: Tuesday, January 16, 2018 9:01 AM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

[Default] On 16 Jan 2018 06:54:52 -0800, in bit.listserv.ibm-main

Post by Ron hawkins
Tim,
Things changed a couple of decades ago.
VSAM physical block is not always the same as the CISZ.
32K CISZ does not waste any space on the track.

Where is this written up? Being retired, I missed this and didn't realize
this was true on the OS390 system I worked on in the 1990s.

Clark Morris

Post by Ron hawkins
Ron
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send

R.S.

2018-01-19 13:36:24 UTC

Permalink

Post by Ron hawkins
Clark,
I don't know where it is written up, but it has been this way as long as I
can remember, which is 5 minutes when I go to the shop, and a bit longer for
IDCAMS.
Following example look at the CISZ(32768) versus the physical record size
(16384). Physical record size is the block size used for each record on the
track, so each CI uses two blocks or records and can span a track boundary.

IMHO the CISZ was up to 32kB for a veeery long time (since VSAM
beginning perhaps).
The most important is one could use any CISZ on any disk geometry,
including 3380 or 3350, or older.
Of course PB (Physical Block) could be different, as well as number or
PB per CI.
--
Radoslaw Skorupka
Lodz, Poland

======================================================================

--
Treść tej wiadomości może zawierać informacje prawnie chronione Banku przeznaczone wyłącznie do użytku służbowego adresata. Odbiorcą może być jedynie jej adresat z wyłączeniem dostępu osób trzecich. Jeżeli nie jesteś adresatem niniejszej wiadomości lub pracownikiem upoważnionym do jej przekazania adresatowi, informujemy, że jej rozpowszechnianie, kopiowanie, rozprowadzanie lub inne działanie o podobnym charakterze jest prawnie zabronione i może być karalne. Jeżeli otrzymałeś tę wiadomość omyłkowo, prosimy niezwłocznie zawiadomić nadawcę wysyłając odpowiedź oraz trwale usunąć tę wiadomość włączając w to wszelkie jej kopie wydrukowane lub zapisane na dysku.

This e-mail may contain legally privileged information of the Bank and is intended solely for business use of the addressee. This e-mail may only be received by the addressee and may not be disclosed to any third parties. If you are not the intended addressee of this e-mail or the employee authorized to forward it to the addressee, be advised that any dissemination, copying, distribution or any other similar activity is legally prohibited and may be punishable. If you received this e-mail by mistake please advise the sender immediately by using the reply facility in your e-mail software and delete permanently this e-mail including any copies of it either printed or saved to hard drive.

mBank S.A. z siedzibą w Warszawie, ul. Senatorska 18, 00-950 Warszawa, www.mBank.pl, e-mail: ***@mBank.plSąd Rejonowy dla m. st. Warszawy XII Wydział Gospodarczy Krajowego Rejestru Sądowego, nr rejestru przedsiębiorców KRS 0000025237, NIP: 526-021-50-88. Według stanu na dzień 01.01.2018 r. kapitał zakładowy mBanku S.A. (w całości wpłacony) wynosi 169.248.488 złotych.

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Seymour J Metz

2018-01-16 18:12:58 UTC

Permalink

When was it ever? Bitsavers is your friend.

--
Shmuel (Seymour J.) Metz
http://mason.gmu.edu/~smetz3

________________________________________
From: IBM Mainframe Discussion List <IBM-***@listserv.ua.edu> on behalf of Ron hawkins <***@SBCGLOBAL.NET>
Sent: Tuesday, January 16, 2018 9:56 AM
To: IBM-***@listserv.ua.edu
Subject: Re: VSAM Performance - CPU reduction

Tim,

Things changed a couple of decades ago.

VSAM physical block is not always the same as the CISZ.

32K CISZ does not waste any space on the track.

Ron

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Massimo Biancucci

2018-01-16 16:26:40 UTC

Permalink

Arun,

in my recent experiences I've got bif improvement using SMB (System Managed
Buffer).

If I'm not wrong both DS are EXTENDED so you could.

Try coding into the DD "AMP=('ACCBIAS=SO')" for the main sequentially read
file and "AMP=('ACCBIAS=DO')" for the second one. At the same time enlarge
the REGION size.

Depending on your HW configuration you could consider using compression on
both DS. It will increase CPU time (or not so much if you've got zEDC) but
could improve elapsed .... depending on CPU contention.

Hope this helps.

Regards.
Massimo

Post by Arun Venkatratnam
Hi All,
We are looking to improve the performance of a COBOL program that
processes 2 VSAM files. The first file is the I/P file and every record
read from the input VSAM file is searched for a matching record in the
other file and a report is written. The program also applies some business
rules while comparing each matching record.
The I/P file is read sequentially while the other file is read in a skip
sequential basis. The test files that were used had 32M records each while
production files have 110M records each.
Attached is the strobe report from the execution of the test job. The test
job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU
minute of execution time.
We are attempting to optimize the VSAM access to these files as it is seen
to take more than 50% of the CPU consumed by this job.
In the 'Attribution of CPU execution time' section, we see that the major
contributors are the components 'QSAM INIT I/O & EXITS' (Module IGZEQBL)
and PARTITION COMMUNICATION.
1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.
Thank you
Arun
------------------------------------------------------------
------------------------------------------------------------
------------------------------------------
1Strobe* PERFORMANCE PROFILE PROGRAMA
01/02/2018 PAGE 42
- #ACE ** ATTRIBUTION OF CPU EXECUTION TIME **
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY---------------------
-----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE
SECTION DESCRIPTION SOLO TOTAL
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
.32 .32
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
IGZEQBL QSAM INIT I/O & EXITS 1.93 1.93
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE
SECTION DESCRIPTION
PROGRAMA PROGRAMA 003522
1.30
1.30
----- -----
3.55 3.55
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY---------------------
-----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE
SECTION DESCRIPTION SOLO TOTAL
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
IGZEQBL QSAM INIT I/O & EXITS 1.84 1.84
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
IGZEQBL CURRMEM QSAM INIT I/O & EXITS 4.34 4.38
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
IGZEQBL DVFILE QSAM INIT I/O & EXITS .03 .03
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
IGZEQBL PREVMEM QSAM INIT I/O & EXITS 22.48 22.51
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE
SECTION DESCRIPTION
PROGRAMA PROGRAMA 003522
IGZCPCO CURRMEM PARTITION COMMUNICATION 4.15 4.19
PROGRAMA PROGRAMA 003522
IGZCPCO IGZEVIO VSAM INPUT/OUTPUT .57 .57
PROGRAMA PROGRAMA 003522
IGZCPCO PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----
50.22 50.32
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

John Abell

2018-01-16 16:37:39 UTC

Permalink

You should see a substantial reduction in I/O with SMB as is my experience.

I use a special DATACLAS in SMS for SMB as is required.

John T. Abell
Tel: 800-295-7608 Option 4
President
International: 1-416-593-5578 Option 4
E-mail: ***@intnlsoftwareproducts.com
Fax: 800-295-7609

International: 1-416-593-5579

International Software Products
www.ispinfo.com

This email may contain confidential and privileged material for the sole use of the intended recipient(s). Any review, use, retention, distribution or disclosure by others is strictly prohibited. If you are not the intended
recipient (or authorized to receive on behalf of the named recipient), please contact the sender by reply email and delete all copies of this message. Also,email is susceptible to data corruption, interception,
tampering, unauthorized amendment and viruses. We only send and receive emails on the basis that we are not liable for any such corruption, interception, tampering, amendment or viruses or any consequence thereof.

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Massimo Biancucci
Sent: Tuesday, January 16, 2018 11:28 AM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: VSAM Performance - CPU reduction

Arun,

in my recent experiences I've got bif improvement using SMB (System Managed Buffer).

If I'm not wrong both DS are EXTENDED so you could.

Try coding into the DD "AMP=('ACCBIAS=SO')" for the main sequentially read file and "AMP=('ACCBIAS=DO')" for the second one. At the same time enlarge the REGION size.

Depending on your HW configuration you could consider using compression on both DS. It will increase CPU time (or not so much if you've got zEDC) but could improve elapsed .... depending on CPU contention.

Hope this helps.

Regards.
Massimo

Post by Arun Venkatratnam
Hi All,
We are looking to improve the performance of a COBOL program that
processes 2 VSAM files. The first file is the I/P file and every
record read from the input VSAM file is searched for a matching record
in the other file and a report is written. The program also applies
some business rules while comparing each matching record.
The I/P file is read sequentially while the other file is read in a
skip sequential basis. The test files that were used had 32M records
each while production files have 110M records each.
Attached is the strobe report from the execution of the test job. The
test job takes nearly 7 CPU minutes and was profiled to capture about
1 CPU minute of execution time.
We are attempting to optimize the VSAM access to these files as it is
seen to take more than 50% of the CPU consumed by this job.
In the 'Attribution of CPU execution time' section, we see that the
major contributors are the components 'QSAM INIT I/O & EXITS' (Module
IGZEQBL) and PARTITION COMMUNICATION.
1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.
Thank you
Arun
------------------------------------------------------------
------------------------------------------------------------
------------------------------------------
1Strobe* PERFORMANCE PROFILE PROGRAMA
01/02/2018 PAGE 42
- #ACE ** ATTRIBUTION OF CPU EXECUTION TIME **
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY---------------------
-----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE
SECTION DESCRIPTION SOLO TOTAL
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
.32 .32
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
IGZEQBL QSAM INIT I/O & EXITS 1.93 1.93
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE
SECTION DESCRIPTION
PROGRAMA PROGRAMA 003522
1.30
1.30
----- -----
3.55 3.55
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY---------------------
-----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE
SECTION DESCRIPTION SOLO TOTAL
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
IGZEQBL QSAM INIT I/O & EXITS 1.84 1.84
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
IGZEQBL CURRMEM QSAM INIT I/O & EXITS 4.34 4.38
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
IGZEQBL DVFILE QSAM INIT I/O & EXITS .03 .03
.LELIB CEEBINIT LE/370 BATCH INIT/TERM
IGZEQBL PREVMEM QSAM INIT I/O & EXITS 22.48 22.51
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE
SECTION DESCRIPTION
PROGRAMA PROGRAMA 003522
IGZCPCO CURRMEM PARTITION COMMUNICATION 4.15 4.19
PROGRAMA PROGRAMA 003522
IGZCPCO IGZEVIO VSAM INPUT/OUTPUT .57 .57
PROGRAMA PROGRAMA 003522
IGZCPCO PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----
50.22 50.32
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Peter Vander Woude

2018-01-17 14:41:12 UTC

Permalink

Arun,

In the IDCAMS listing, I see that the files are defined NONSPANNED. With the records being variable length, you could have as few as 2 records per ci, since the max is around 11000. if all the records are the minimum of 170, then you would have around 156 records per ci.

Have you looked at changing NONSPANNED to SPANNED? That will allow variable length records to go across CI's, and might help in some small way, by decreasing the actual space that the files use up. As such the # of records read in, as defined by the BUFND, will be higher.

for the skip sequential file, if you are doing a point and then read next, what is the average number of records that are being processed? Having the BUFND so high, if it's only processing a few records in the data component, then you are reading in, and spending higher cpu time, just to throw away those records and read in the next set of ci's. Anytime you reduce the BUFND, of course your EXCP's will go up, as VSAM I/O counts are the number of times it had to read a group of ci's, as opposed to sequential files, where each block counts as an I/O, no matter what BUFNO is set to.

Peter

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Ron hawkins

2018-01-19 13:43:36 UTC

Permalink

Peter,

Spanned records on VSAM are not the same as DSORG=PS.

I need to verify this, but my memory says that even with spanned records, VSAM will write a record in a new CI if it will not fit in the current CI. I thought that spanned records are only written when the record is greater than the CISZ.

That means spanned records space efficiency with VSAM depends on the number of records greater than the CISZ. I think that is why it is good for SMF ESDS.

Ron

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Peter Vander Woude
Sent: Wednesday, January 17, 2018 6:43 AM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

Arun,

In the IDCAMS listing, I see that the files are defined NONSPANNED. With the records being variable length, you could have as few as 2 records per ci, since the max is around 11000. if all the records are the minimum of 170, then you would have around 156 records per ci.

Have you looked at changing NONSPANNED to SPANNED? That will allow variable length records to go across CI's, and might help in some small way, by decreasing the actual space that the files use up. As such the # of records read in, as defined by the BUFND, will be higher.

for the skip sequential file, if you are doing a point and then read next, what is the average number of records that are being processed? Having the BUFND so high, if it's only processing a few records in the data component, then you are reading in, and spending higher cpu time, just to throw away those records and read in the next set of ci's. Anytime you reduce the BUFND, of course your EXCP's will go up, as VSAM I/O counts are the number of times it had to read a group of ci's, as opposed to sequential files, where each block counts as an I/O, no matter what BUFNO is set to.

Peter

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Tom Marchant

2018-01-19 15:10:02 UTC

Permalink

Post by R.S.
IMHO the CISZ was up to 32kB for a veeery long time (since VSAM
beginning perhaps).
The most important is one could use any CISZ on any disk geometry,
including 3380 or 3350, or older.
Of course PB (Physical Block) could be different, as well as number or
PB per CI.

--
Tom Marchant

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Ron hawkins

2018-01-22 14:50:27 UTC

Permalink

Great detective work.

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On Behalf Of Tom Marchant
Sent: Friday, January 19, 2018 7:11 AM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

The OS/MVS Overview manual, GC28-0984-0, dated June, 1978, is available on bitsavers at http://bitsavers.trailing-edge.com/pdf/ibm/370/OS_VS2/Release_3.7_1977/GC28-0984-0_OS_VS2_MVS_Overview_Rel_3.7_Jun78.pdf

On pages 8-18 and 8-19 (151-152) talks about control intervals and has this:

<quote>
The size of the control interval need not correspond to the size of a track on the device. Figure 8.9 shows the independence of control intervals from physical records, which are limited by the capacity of a track on a particular device.
</quote>

This confirms Ron's and Radoslaw's memory.

--
Tom Marchant

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN

Clark Morris

2018-01-24 00:55:47 UTC

Permalink

[Default] On 5 Jan 2018 16:28:48 -0800, in bit.listserv.ibm-main

I assume using random access for the second file was tried with
adequate buffering for index and data. BLSR or the more current means
of doing random access buffering should have been used. It may also
help to save any randomly read record that were read based on
information in records from the second file based on the match with a
record from the first file. Knowing access patterns can help in
determining the best solution.

Clark Morris

Post by Arun Venkatratnam
Attached is the strobe report from the execution of the test job. The test job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU minute of execution time.
We are attempting to optimize the VSAM access to these files as it is seen to take more than 50% of the CPU consumed by this job.
In the 'Attribution of CPU execution time' section, we see that the major contributors are the components 'QSAM INIT I/O & EXITS' (Module IGZEQBL) and PARTITION COMMUNICATION.
1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.
Thank you
Arun
------------------------------------------------------------------------------------------------------------------------------------------------------------------
1Strobe* PERFORMANCE PROFILE PROGRAMA 01/02/2018 PAGE 42
- #ACE ** ATTRIBUTION OF CPU EXECUTION TIME **
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY--------------------- -----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE SECTION DESCRIPTION SOLO TOTAL
.LELIB CEEBINIT LE/370 BATCH INIT/TERM .32 .32
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL QSAM INIT I/O & EXITS 1.93 1.93
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE SECTION DESCRIPTION
PROGRAMA PROGRAMA 003522 1.30 1.30
----- -----
3.55 3.55
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY--------------------- -----------------VIA----------------------- CPU TIME %
XACTION MODULE SECTION DESCRIPTION MODULE SECTION DESCRIPTION SOLO TOTAL
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL QSAM INIT I/O & EXITS 1.84 1.84
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL CURRMEM QSAM INIT I/O & EXITS 4.34 4.38
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL DVFILE QSAM INIT I/O & EXITS .03 .03
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL PREVMEM QSAM INIT I/O & EXITS 22.48 22.51
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE SECTION DESCRIPTION
PROGRAMA PROGRAMA 003522 IGZCPCO CURRMEM PARTITION COMMUNICATION 4.15 4.19
PROGRAMA PROGRAMA 003522 IGZCPCO IGZEVIO VSAM INPUT/OUTPUT .57 .57
PROGRAMA PROGRAMA 003522 IGZCPCO PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----
50.22 50.32
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

Ron hawkins

2018-01-24 01:35:50 UTC

Permalink

Clark,

If you had time to read through this lengthy thread you will find that the
2nd file uses skip-sequential access. LSR is usually not an appropriate
strategy for this access pattern.

The OP has tried reducing BUFND on the second file, and observed a reduction
in throughput, which verifies the extent to which the sequential access is
taking advantage of chained Cis.

Ron

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On
Behalf Of Clark Morris
Sent: Tuesday, January 23, 2018 4:57 PM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

[Default] On 5 Jan 2018 16:28:48 -0800, in bit.listserv.ibm-main

Post by Arun Venkatratnam
Hi All,
We are looking to improve the performance of a COBOL program that processes

2 VSAM files. The first file is the I/P file and every record read from the
input VSAM file is searched for a matching record in the other file and a
report is written. The program also applies some business rules while
comparing each matching record.

Post by Arun Venkatratnam
The I/P file is read sequentially while the other file is read in a skip

sequential basis. The test files that were used had 32M records each while
production files have 110M records each.

I assume using random access for the second file was tried with adequate
buffering for index and data. BLSR or the more current means of doing random
access buffering should have been used. It may also help to save any
randomly read record that were read based on information in records from the
second file based on the match with a record from the first file. Knowing
access patterns can help in determining the best solution.

Clark Morris

Post by Arun Venkatratnam
Attached is the strobe report from the execution of the test job. The test

job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU
minute of execution time.

Post by Arun Venkatratnam
We are attempting to optimize the VSAM access to these files as it is seen

to take more than 50% of the CPU consumed by this job.

Post by Arun Venkatratnam
In the 'Attribution of CPU execution time' section, we see that the major

contributors are the components 'QSAM INIT I/O & EXITS' (Module IGZEQBL)
and PARTITION COMMUNICATION.

Post by Arun Venkatratnam
1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.
Thank you
Arun
-----------------------------------------------------------------------
-----------------------------------------------------------------------
--------------------
1Strobe* PERFORMANCE PROFILE PROGRAMA

01/02/2018 PAGE 42

Post by Arun Venkatratnam
- #ACE ** ATTRIBUTION OF CPU EXECUTION TIME **
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

Post by Arun Venkatratnam
XACTION MODULE SECTION DESCRIPTION MODULE

SECTION DESCRIPTION SOLO TOTAL

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM

.32 32

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

QSAM INIT I/O & EXITS 1.93 1.93

Post by Arun Venkatratnam
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE

SECTION DESCRIPTION

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522

1.30 1.30
----- -----
3.55 3.55

Post by Arun Venkatratnam
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

Post by Arun Venkatratnam
XACTION MODULE SECTION DESCRIPTION MODULE

SECTION DESCRIPTION SOLO TOTAL

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

QSAM INIT I/O & EXITS 1.84 1.84

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

CURRMEM QSAM INIT I/O & EXITS 4.34 4.38

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

DVFILE QSAM INIT I/O & EXITS .03 .03

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

PREVMEM QSAM INIT I/O & EXITS 22.48 22.51

Post by Arun Venkatratnam
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE

SECTION DESCRIPTION

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

CURRMEM PARTITION COMMUNICATION 4.15 4.19

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

IGZEVIO VSAM INPUT/OUTPUT .57 .57

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----

Post by Arun Venkatratnam
50.22 50.32
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send

Clark Morris

2018-01-24 12:49:13 UTC

Permalink

[Default] On 23 Jan 2018 17:35:50 -0800, in bit.listserv.ibm-main

Post by Ron hawkins
Clark,
If you had time to read through this lengthy thread you will find that the
2nd file uses skip-sequential access. LSR is usually not an appropriate
strategy for this access pattern.

I realize he was using skip sequential. My point was that random
access looks like it could be a better fit for this file than skip
sequential especially since a second record may have to be read after
the first record is found on the second file.

Clark Morris

Post by Ron hawkins
The OP has tried reducing BUFND on the second file, and observed a reduction
in throughput, which verifies the extent to which the sequential access is
taking advantage of chained Cis.
Ron
-----Original Message-----
Behalf Of Clark Morris
Sent: Tuesday, January 23, 2018 4:57 PM
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction
[Default] On 5 Jan 2018 16:28:48 -0800, in bit.listserv.ibm-main

Post by Arun Venkatratnam
Hi All,
We are looking to improve the performance of a COBOL program that processes

Post by Arun Venkatratnam
The I/P file is read sequentially while the other file is read in a skip

sequential basis. The test files that were used had 32M records each while
production files have 110M records each.
I assume using random access for the second file was tried with adequate
buffering for index and data. BLSR or the more current means of doing random
access buffering should have been used. It may also help to save any
randomly read record that were read based on information in records from the
second file based on the match with a record from the first file. Knowing
access patterns can help in determining the best solution.
Clark Morris

Post by Arun Venkatratnam
Attached is the strobe report from the execution of the test job. The test

job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU
minute of execution time.

Post by Arun Venkatratnam
We are attempting to optimize the VSAM access to these files as it is seen

to take more than 50% of the CPU consumed by this job.

Post by Arun Venkatratnam
In the 'Attribution of CPU execution time' section, we see that the major

contributors are the components 'QSAM INIT I/O & EXITS' (Module IGZEQBL)
and PARTITION COMMUNICATION.

01/02/2018 PAGE 42

Post by Arun Venkatratnam
- #ACE ** ATTRIBUTION OF CPU EXECUTION

TIME **

Post by Arun Venkatratnam
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

Post by Arun Venkatratnam
XACTION MODULE SECTION DESCRIPTION MODULE

SECTION DESCRIPTION SOLO TOTAL

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM

.32 32

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

QSAM INIT I/O & EXITS 1.93 1.93

Post by Arun Venkatratnam
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE

SECTION DESCRIPTION

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522

1.30 1.30
----- -----
3.55 3.55

Post by Arun Venkatratnam
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

Post by Arun Venkatratnam
XACTION MODULE SECTION DESCRIPTION MODULE

SECTION DESCRIPTION SOLO TOTAL

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

QSAM INIT I/O & EXITS 1.84 1.84

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

CURRMEM QSAM INIT I/O & EXITS 4.34 4.38

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

DVFILE QSAM INIT I/O & EXITS .03 .03

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

PREVMEM QSAM INIT I/O & EXITS 22.48 22.51

Post by Arun Venkatratnam
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE

SECTION DESCRIPTION

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

CURRMEM PARTITION COMMUNICATION 4.15 4.19

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

IGZEVIO VSAM INPUT/OUTPUT .57 .57

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----

Post by Arun Venkatratnam
50.22 50.32
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

Ron hawkins

2018-01-24 17:53:39 UTC

Permalink

Clark,

It's not that the process is reading a 2nd record in the same CI. That would
result in a buffer hit irrespective of whether it is LSR or NSR.

The empirical evidence of his test with BUFND=2 reduced from BUFND=240 is
that the program reads more than one CI sequentially for each skip. That
alone makes the file a poor candidate for LSR.

Ron

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On
Behalf Of Clark Morris
Sent: Wednesday, January 24, 2018 4:50 AM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

[Default] On 23 Jan 2018 17:35:50 -0800, in bit.listserv.ibm-main

Post by Ron hawkins
Clark,
If you had time to read through this lengthy thread you will find that
the 2nd file uses skip-sequential access. LSR is usually not an
appropriate strategy for this access pattern.

I realize he was using skip sequential. My point was that random access
looks like it could be a better fit for this file than skip sequential
especially since a second record may have to be read after the first record
is found on the second file.

Clark Morris

Post by Ron hawkins
The OP has tried reducing BUFND on the second file, and observed a
reduction in throughput, which verifies the extent to which the
sequential access is taking advantage of chained Cis.
Ron
-----Original Message-----
On Behalf Of Clark Morris
Sent: Tuesday, January 23, 2018 4:57 PM
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction
[Default] On 5 Jan 2018 16:28:48 -0800, in bit.listserv.ibm-main

Post by Arun Venkatratnam
Hi All,
We are looking to improve the performance of a COBOL program that processes

2 VSAM files. The first file is the I/P file and every record read from
the input VSAM file is searched for a matching record in the other file
and a report is written. The program also applies some business rules
while comparing each matching record.

Post by Arun Venkatratnam
The I/P file is read sequentially while the other file is read in a skip

sequential basis. The test files that were used had 32M records each
while production files have 110M records each.
I assume using random access for the second file was tried with
adequate buffering for index and data. BLSR or the more current means
of doing random access buffering should have been used. It may also
help to save any randomly read record that were read based on
information in records from the second file based on the match with a
record from the first file. Knowing access patterns can help in

determining the best solution.

Post by Ron hawkins
Clark Morris

Post by Arun Venkatratnam
Attached is the strobe report from the execution of the test job. The test

job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU
minute of execution time.

Post by Arun Venkatratnam
We are attempting to optimize the VSAM access to these files as it is seen

to take more than 50% of the CPU consumed by this job.

Post by Arun Venkatratnam
In the 'Attribution of CPU execution time' section, we see that the major

contributors are the components 'QSAM INIT I/O & EXITS' (Module
IGZEQBL) and PARTITION COMMUNICATION.

Post by Arun Venkatratnam
1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.
Thank you
Arun
----------------------------------------------------------------------
-
----------------------------------------------------------------------
-
--------------------
1Strobe* PERFORMANCE PROFILE PROGRAMA

01/02/2018 PAGE 42

Post by Arun Venkatratnam
- #ACE ** ATTRIBUTION OF CPU EXECUTION

TIME **

Post by Arun Venkatratnam
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

Post by Arun Venkatratnam
XACTION MODULE SECTION DESCRIPTION MODULE

SECTION DESCRIPTION SOLO TOTAL

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM

.32 32

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

QSAM INIT I/O & EXITS 1.93 1.93

Post by Arun Venkatratnam
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE

SECTION DESCRIPTION

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522

1.30 1.30
----- -----
3.55 3.55

Post by Arun Venkatratnam
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

Post by Arun Venkatratnam
XACTION MODULE SECTION DESCRIPTION MODULE

SECTION DESCRIPTION SOLO TOTAL

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

QSAM INIT I/O & EXITS 1.84 1.84

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

CURRMEM QSAM INIT I/O & EXITS 4.34 4.38

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

DVFILE QSAM INIT I/O & EXITS .03 .03

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

PREVMEM QSAM INIT I/O & EXITS 22.48 22.51

Post by Arun Venkatratnam
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE

SECTION DESCRIPTION

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

CURRMEM PARTITION COMMUNICATION 4.15 4.19

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

IGZEVIO VSAM INPUT/OUTPUT .57 .57

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----

Post by Arun Venkatratnam
50.22 50.32
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send

Clark Morris

2018-01-25 01:44:57 UTC

Permalink

[Default] On 24 Jan 2018 09:53:39 -0800, in bit.listserv.ibm-main

Post by Ron hawkins
Clark,
It's not that the process is reading a 2nd record in the same CI. That would
result in a buffer hit irrespective of whether it is LSR or NSR.
The empirical evidence of his test with BUFND=2 reduced from BUFND=240 is
that the program reads more than one CI sequentially for each skip. That
alone makes the file a poor candidate for LSR.

The original poster also said that under certain circumstances after
reading a record skip sequential on the second file by finding a
match, under x conditions another record was retrieved from the same
file. Depending on the frequency of this condition occurring and the
access pattern LSR might help for that condition. Program caching of
hits and misses may be more appropriate depending on circumstance. I
had a case in one shop where thousand of read not found conditions
occurred for about 12 records. This was a table file and around 20
years ago so the exact details escape me but the point is that much
depends on the overall processing.

Clark Morris

Post by Ron hawkins
Ron
-----Original Message-----
Behalf Of Clark Morris
Sent: Wednesday, January 24, 2018 4:50 AM
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction
[Default] On 23 Jan 2018 17:35:50 -0800, in bit.listserv.ibm-main

Post by Ron hawkins
Clark,
If you had time to read through this lengthy thread you will find that
the 2nd file uses skip-sequential access. LSR is usually not an
appropriate strategy for this access pattern.

I realize he was using skip sequential. My point was that random access
looks like it could be a better fit for this file than skip sequential
especially since a second record may have to be read after the first record
is found on the second file.
Clark Morris

Post by Ron hawkins
The OP has tried reducing BUFND on the second file, and observed a
reduction in throughput, which verifies the extent to which the
sequential access is taking advantage of chained Cis.
Ron
-----Original Message-----
On Behalf Of Clark Morris
Sent: Tuesday, January 23, 2018 4:57 PM
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction
[Default] On 5 Jan 2018 16:28:48 -0800, in bit.listserv.ibm-main

Post by Arun Venkatratnam
Hi All,
We are looking to improve the performance of a COBOL program that processes

2 VSAM files. The first file is the I/P file and every record read from
the input VSAM file is searched for a matching record in the other file
and a report is written. The program also applies some business rules
while comparing each matching record.

Post by Arun Venkatratnam
The I/P file is read sequentially while the other file is read in a skip

sequential basis. The test files that were used had 32M records each
while production files have 110M records each.
I assume using random access for the second file was tried with
adequate buffering for index and data. BLSR or the more current means
of doing random access buffering should have been used. It may also
help to save any randomly read record that were read based on
information in records from the second file based on the match with a
record from the first file. Knowing access patterns can help in

determining the best solution.

Post by Ron hawkins
Clark Morris

Post by Arun Venkatratnam
Attached is the strobe report from the execution of the test job. The test

job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU
minute of execution time.

Post by Arun Venkatratnam
We are attempting to optimize the VSAM access to these files as it is seen

to take more than 50% of the CPU consumed by this job.

Post by Arun Venkatratnam
In the 'Attribution of CPU execution time' section, we see that the major

contributors are the components 'QSAM INIT I/O & EXITS' (Module
IGZEQBL) and PARTITION COMMUNICATION.

Post by Arun Venkatratnam
1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.
Thank you
Arun
----------------------------------------------------------------------
-
----------------------------------------------------------------------
-
--------------------
1Strobe* PERFORMANCE PROFILE PROGRAMA

01/02/2018 PAGE 42

Post by Arun Venkatratnam
- #ACE ** ATTRIBUTION OF CPU EXECUTION

TIME **

Post by Arun Venkatratnam
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

Post by Arun Venkatratnam
XACTION MODULE SECTION DESCRIPTION MODULE

SECTION DESCRIPTION SOLO TOTAL

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM

.32 32

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

QSAM INIT I/O & EXITS 1.93 1.93

Post by Arun Venkatratnam
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE

SECTION DESCRIPTION

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522

1.30 1.30
----- -----
3.55 3.55

Post by Arun Venkatratnam
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

Post by Arun Venkatratnam
XACTION MODULE SECTION DESCRIPTION MODULE

SECTION DESCRIPTION SOLO TOTAL

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

QSAM INIT I/O & EXITS 1.84 1.84

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

CURRMEM QSAM INIT I/O & EXITS 4.34 4.38

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

DVFILE QSAM INIT I/O & EXITS .03 .03

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM IGZEQBL

PREVMEM QSAM INIT I/O & EXITS 22.48 22.51

Post by Arun Venkatratnam
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE

SECTION DESCRIPTION

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

CURRMEM PARTITION COMMUNICATION 4.15 4.19

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

IGZEVIO VSAM INPUT/OUTPUT .57 .57

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522 IGZCPCO

PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----

Post by Arun Venkatratnam
50.22 50.32
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send

Ron hawkins

2018-01-25 10:49:26 UTC

Permalink

Clark,

it is a 15GB KSDS, with 564,000 Cis, and 32,448,318 records.

It is very optimistic to think that after he verified increased IO when
reading one CI, it will be more than offset LRU buffer hits when there is
bursts of sequential IO walking the buffer pool a cylinder at a time.

NSR behavior will not let the random hit that you describe occur outside of
one CI in the chain as COBOL only uses one string. The program can read all
the other CI's in, but the program only looks at them in sequential mode. In
skip (random) it only looks at the CI currently pointed to by the string,
and ignores everything else in memory.

What sort of buffers were you thinking of? 5GB worth?

Ron

-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-***@LISTSERV.UA.EDU] On
Behalf Of Clark Morris
Sent: Wednesday, January 24, 2018 5:46 PM
To: IBM-***@LISTSERV.UA.EDU
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction

[Default] On 24 Jan 2018 09:53:39 -0800, in bit.listserv.ibm-main

Post by Ron hawkins
Clark,
It's not that the process is reading a 2nd record in the same CI. That
would result in a buffer hit irrespective of whether it is LSR or NSR.
The empirical evidence of his test with BUFND=2 reduced from BUFND=240
is that the program reads more than one CI sequentially for each skip.
That alone makes the file a poor candidate for LSR.

The original poster also said that under certain circumstances after reading
a record skip sequential on the second file by finding a match, under x
conditions another record was retrieved from the same file. Depending on
the frequency of this condition occurring and the access pattern LSR might
help for that condition. Program caching of hits and misses may be more
appropriate depending on circumstance. I had a case in one shop where
thousand of read not found conditions occurred for about 12 records. This
was a table file and around 20 years ago so the exact details escape me but
the point is that much depends on the overall processing.

Clark Morris

Post by Ron hawkins
Ron
-----Original Message-----
On Behalf Of Clark Morris
Sent: Wednesday, January 24, 2018 4:50 AM
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction
[Default] On 23 Jan 2018 17:35:50 -0800, in bit.listserv.ibm-main

Post by Ron hawkins
Clark,
If you had time to read through this lengthy thread you will find that
the 2nd file uses skip-sequential access. LSR is usually not an
appropriate strategy for this access pattern.

Post by Ron hawkins
The OP has tried reducing BUFND on the second file, and observed a
reduction in throughput, which verifies the extent to which the
sequential access is taking advantage of chained Cis.
Ron
-----Original Message-----
On Behalf Of Clark Morris
Sent: Tuesday, January 23, 2018 4:57 PM
Subject: Re: [IBM-MAIN] VSAM Performance - CPU reduction
[Default] On 5 Jan 2018 16:28:48 -0800, in bit.listserv.ibm-main

Post by Arun Venkatratnam
Hi All,
We are looking to improve the performance of a COBOL program that processes

2 VSAM files. The first file is the I/P file and every record read
from the input VSAM file is searched for a matching record in the
other file and a report is written. The program also applies some
business rules while comparing each matching record.

Post by Arun Venkatratnam
The I/P file is read sequentially while the other file is read in a skip

sequential basis. The test files that were used had 32M records each
while production files have 110M records each.
I assume using random access for the second file was tried with
adequate buffering for index and data. BLSR or the more current means
of doing random access buffering should have been used. It may also
help to save any randomly read record that were read based on
information in records from the second file based on the match with a
record from the first file. Knowing access patterns can help in

determining the best solution.

Post by Ron hawkins
Clark Morris

Post by Arun Venkatratnam
Attached is the strobe report from the execution of the test job. The test

job takes nearly 7 CPU minutes and was profiled to capture about 1 CPU
minute of execution time.

Post by Arun Venkatratnam
We are attempting to optimize the VSAM access to these files as it is seen

to take more than 50% of the CPU consumed by this job.

Post by Arun Venkatratnam
In the 'Attribution of CPU execution time' section, we see that the major

contributors are the components 'QSAM INIT I/O & EXITS' (Module
IGZEQBL) and PARTITION COMMUNICATION.

Post by Arun Venkatratnam
1.What these components are
2.Why is QSAM access used instead of VSAM I/O access.
3.What needs to be done to reduce the CPU consumption by these components.
Thank you
Arun
---------------------------------------------------------------------
-
-
---------------------------------------------------------------------
-
-
--------------------
1Strobe* PERFORMANCE PROFILE PROGRAMA

01/02/2018 PAGE 42

Post by Arun Venkatratnam
- #ACE ** ATTRIBUTION OF CPU EXECUTION

TIME **

Post by Arun Venkatratnam
-.COBLIB IGZCPCO IGZEVIO VSAM INPUT/OUTPUT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

Post by Arun Venkatratnam
XACTION MODULE SECTION DESCRIPTION MODULE

SECTION DESCRIPTION SOLO TOTAL

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM

.32 32

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM

IGZEQBL

Post by Ron hawkins

Post by Ron hawkins
QSAM INIT I/O & EXITS 1.93 1.93

Post by Arun Venkatratnam
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE

SECTION DESCRIPTION

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522

1.30 1.30
----- -----
3.55 3.55

Post by Arun Venkatratnam
-.VSAM IDA019L1 VSAM RECORD MANAGEMENT
---------------------------WAS INVOKED BY---------------------

-----------------VIA----------------------- CPU TIME %

Post by Arun Venkatratnam
XACTION MODULE SECTION DESCRIPTION MODULE

SECTION DESCRIPTION SOLO TOTAL

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM

IGZEQBL

Post by Ron hawkins

Post by Ron hawkins
QSAM INIT I/O & EXITS 1.84 1.84

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM

IGZEQBL

Post by Ron hawkins

Post by Ron hawkins
CURRMEM QSAM INIT I/O & EXITS 4.34 4.38

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM

IGZEQBL

Post by Ron hawkins

Post by Ron hawkins
DVFILE QSAM INIT I/O & EXITS .03 .03

Post by Arun Venkatratnam
.LELIB CEEBINIT LE/370 BATCH INIT/TERM

IGZEQBL

Post by Ron hawkins

Post by Ron hawkins
PREVMEM QSAM INIT I/O & EXITS 22.48 22.51

Post by Arun Venkatratnam
XACTION MODULE SECTION LOCATION LINE SOURCE TEXT MODULE

SECTION DESCRIPTION

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522

IGZCPCO

Post by Ron hawkins

Post by Ron hawkins
CURRMEM PARTITION COMMUNICATION 4.15 4.19

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522

IGZCPCO

Post by Ron hawkins

Post by Ron hawkins
IGZEVIO VSAM INPUT/OUTPUT .57 .57

Post by Arun Venkatratnam
PROGRAMA PROGRAMA 003522

IGZCPCO

Post by Ron hawkins

Post by Ron hawkins
PREVMEM PARTITION COMMUNICATION 16.80 16.80
----- -----

Post by Arun Venkatratnam
50.22 50.32
---------------------------------------------------------------------
- For IBM-MAIN subscribe / signoff / archive access instructions,
IBM-MAIN