Discussion:
ADABAS tuning
(too old to reply)
Tony Thigpen
2018-08-02 15:14:10 UTC
Permalink
Is there anyone that would be willing to give me some guidance to solve
some ADABAS issues. Something is causing a performance problem and
nobody seems to even know where to start looking.

We have several Natural programs that run in minutes one night, then
take an hour on other nights. We also have some RPG programs (using the
VSAM bridge) that have similar problems.

The programmers don't think it has to do with daily work-load changes.
They want to point the finger at Systems. We (systems) are not making
changes.

The 'DBA' belongs to the programmer group, and to be honest, I don't
think he has a clue concerning performance, just how to define the
databases.

At this point, I just need a better idea of where the problem really is
so I can attempt to throw it back to the DBA.
--
Tony Thigpen

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Rob Schramm
2018-08-02 15:16:29 UTC
Permalink
Call support


Rob Schramm
Post by Tony Thigpen
Is there anyone that would be willing to give me some guidance to solve
some ADABAS issues. Something is causing a performance problem and
nobody seems to even know where to start looking.
We have several Natural programs that run in minutes one night, then
take an hour on other nights. We also have some RPG programs (using the
VSAM bridge) that have similar problems.
The programmers don't think it has to do with daily work-load changes.
They want to point the finger at Systems. We (systems) are not making
changes.
The 'DBA' belongs to the programmer group, and to be honest, I don't
think he has a clue concerning performance, just how to define the
databases.
At this point, I just need a better idea of where the problem really is
so I can attempt to throw it back to the DBA.
--
Tony Thigpen
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
--
Rob Schramm

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Carmen Vitullo
2018-08-02 15:27:45 UTC
Permalink
I worked on a system (state govm't) that had ADABAS years ago, mostly it depends on what version your running, some tuning opportunities were in the latest release, about 2011,2012 timeframe, another tuning opportunity was placement of the DB's and index files( ASSOC file) - what I recall back with DB performance was DASD CACHE, it was suggested DFW was turned off, IIRC, and the ASSOC file and DB be on the same volume? that's all from memory, a good resource as Rob has suggested is ADABAS support.



Carmen Vitullo

----- Original Message -----

From: "Tony Thigpen" <***@VSE2PDF.COM>
To: IBM-***@LISTSERV.UA.EDU
Sent: Thursday, August 2, 2018 10:13:58 AM
Subject: ADABAS tuning

Is there anyone that would be willing to give me some guidance to solve
some ADABAS issues. Something is causing a performance problem and
nobody seems to even know where to start looking.

We have several Natural programs that run in minutes one night, then
take an hour on other nights. We also have some RPG programs (using the
VSAM bridge) that have similar problems.

The programmers don't think it has to do with daily work-load changes.
They want to point the finger at Systems. We (systems) are not making
changes.

The 'DBA' belongs to the programmer group, and to be honest, I don't
think he has a clue concerning performance, just how to define the
databases.

At this point, I just need a better idea of where the problem really is
so I can attempt to throw it back to the DBA.
--
Tony Thigpen

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN


----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
ITschak Mugzach
2018-08-02 15:32:19 UTC
Permalink
Is cpu consumption equal or near? It might a wait for a reaource. Don't u
have a monitor to look at?

ITschak
Post by Carmen Vitullo
I worked on a system (state govm't) that had ADABAS years ago, mostly it
depends on what version your running, some tuning opportunities were in the
latest release, about 2011,2012 timeframe, another tuning opportunity was
placement of the DB's and index files( ASSOC file) - what I recall back
with DB performance was DASD CACHE, it was suggested DFW was turned off,
IIRC, and the ASSOC file and DB be on the same volume? that's all from
memory, a good resource as Rob has suggested is ADABAS support.
Carmen Vitullo
----- Original Message -----
Sent: Thursday, August 2, 2018 10:13:58 AM
Subject: ADABAS tuning
Is there anyone that would be willing to give me some guidance to solve
some ADABAS issues. Something is causing a performance problem and
nobody seems to even know where to start looking.
We have several Natural programs that run in minutes one night, then
take an hour on other nights. We also have some RPG programs (using the
VSAM bridge) that have similar problems.
The programmers don't think it has to do with daily work-load changes.
They want to point the finger at Systems. We (systems) are not making
changes.
The 'DBA' belongs to the programmer group, and to be honest, I don't
think he has a clue concerning performance, just how to define the
databases.
At this point, I just need a better idea of where the problem really is
so I can attempt to throw it back to the DBA.
--
Tony Thigpen
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Brian France
2018-08-02 15:30:24 UTC
Permalink
How about from a WLM perspective or use RMF to view the times they run
as it might show the problems. RMF III you can watch it live.

I believe ADABAS has info in it's address space that shows file usage.
Now that might only show when the address space is shutdown.
Post by Tony Thigpen
Is there anyone that would be willing to give me some guidance to
solve some ADABAS issues. Something is causing a performance problem
and nobody seems to even know where to start looking.
We have several Natural programs that run in minutes one night, then
take an hour on other nights. We also have some RPG programs (using
the VSAM bridge) that have similar problems.
The programmers don't think it has to do with daily work-load changes.
They want to point the finger at Systems. We (systems) are not making
changes.
The 'DBA' belongs to the programmer group, and to be honest, I don't
think he has a clue concerning performance, just how to define the
databases.
At this point, I just need a better idea of where the problem really
is so I can attempt to throw it back to the DBA.
--
Brian W. France
Systems Administrator (Mainframe)
Pennsylvania State University
Administrative Information Services - Infrastructure/SYSARC
Rm 25 Shields Bldg., University Park, Pa. 16802
814-863-4739
***@psu.edu

"To make an apple pie from scratch, you must first invent the universe."

Carl Sagan

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Elardus Engelbrecht
2018-08-02 15:42:42 UTC
Permalink
Is there anyone that would be willing to give me some guidance to solve some ADABAS issues. Something is causing a performance problem and nobody seems to even know where to start looking.
At what z/OS and ADABAS level are you?
We have several Natural programs that run in minutes one night, then take an hour on other nights. We also have some RPG programs (using the VSAM bridge) that have similar problems.
Batch or Online? In what WLM classes / priority are these programs running? What is RMF telling you about performance?

Are the _SAME_ transactions / ADABAS calls being run in one night and then longer in other nights?

Are the duration not perhaps being due to enqueues or lockouts inside the database?
The programmers don't think it has to do with daily work-load changes. They want to point the finger at Systems. We (systems) are not making changes.
Ask them since when does it happen. Also, how often. What was done earlier in the day before the nights where it took longer?

Ask them how often do they re-organizing / enlarging their database(s). This is important.
The 'DBA' belongs to the programmer group, and to be honest, I don't think he has a clue concerning performance, just how to define the databases.
Are the Com-Plete records written out? These records will tell you a lot about ADABAS transactions, calls and workload.

Check your Buffer Pool, Work Pool, etc. There are many tuning settings inside ADABAS and Com-Plete. Check them out.
At this point, I just need a better idea of where the problem really is so I can attempt to throw it back to the DBA.
I think there is a SAG-L discussion list somewhere, but you should call Software AG for support.

Trust me, ADABAS tuning is a thankless job.

Groete / Greetings
Elardus Engelbrecht

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Pew, Curtis G
2018-08-02 16:29:21 UTC
Permalink
Post by Elardus Engelbrecht
I think there is a SAG-L discussion list somewhere
SAG-***@LISTSERV.UARK.EDU

Software AG also has discussion forums, documentation, and such at empower.softwareag.com; you might be able to get an account there without the DBA’s permission (and I think if you did you could open support issues.)
--
Pew, Curtis G
***@austin.utexas.edu
ITS Systems/Core/Administrative Services


----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Stephen Donaldson
2018-08-02 15:43:06 UTC
Permalink
Hi,   If you're not getting anywhere and need some consulting then
consider calling us.  Details below.

Regards,
Stephen Donaldson.
--
Stephen R. Donaldson, Code Magus Limited (England reg. no. 4024745)
Number 6, 69 Woodstock Road, Oxford, OX2 6EY, United Kingdom Voice: +44
1865 310 768 Fax: +44 1865 316 979 Cell: +44 787 9897709 Support: +44
1865 589826 Skype:vixxmovz http://www.codemagus.com
Post by Tony Thigpen
Is there anyone that would be willing to give me some guidance to
solve some ADABAS issues. Something is causing a performance problem
and nobody seems to even know where to start looking.
We have several Natural programs that run in minutes one night, then
take an hour on other nights. We also have some RPG programs (using
the VSAM bridge) that have similar problems.
The programmers don't think it has to do with daily work-load changes.
They want to point the finger at Systems. We (systems) are not making
changes.
The 'DBA' belongs to the programmer group, and to be honest, I don't
think he has a clue concerning performance, just how to define the
databases.
At this point, I just need a better idea of where the problem really
is so I can attempt to throw it back to the DBA.
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Tony Thigpen
2018-08-02 15:43:32 UTC
Permalink
This post might be inappropriate. Click to display it.
Clark Morris
2018-08-02 18:40:39 UTC
Permalink
[Default] On 2 Aug 2018 08:43:32 -0700, in bit.listserv.ibm-main
Post by Tony Thigpen
1) There is no monitor.
2) WLM only has two profiles, DAY and NIGHT. These jobs all run during
the NIGHT settings.
3) Turf war prevents me from actually calling ADABAS support. Only the
DBA is allowed that option. And, if the issues are resolved with ADABAS
tuning, then he will look really bad as he has placed all the blame on
'Systems'. So, he will not call them for us.
I would compare all of the job statistics that you have access to
including CPU time and i-os to see what differs between normal and
spike nights. Also check what is running concurrently with the spike
night jobs. It may be a scheduling conflict. Also if Adabas is
cutting any SMF records that could be helpful get those and ask for
access to the tool(s) to interpret them.

Clark Morris
Post by Tony Thigpen
I need to get enough ammo to dump it back on the DBA.
Tony Thigpen
Post by Tony Thigpen
Is there anyone that would be willing to give me some guidance to solve
some ADABAS issues. Something is causing a performance problem and
nobody seems to even know where to start looking.
We have several Natural programs that run in minutes one night, then
take an hour on other nights. We also have some RPG programs (using the
VSAM bridge) that have similar problems.
The programmers don't think it has to do with daily work-load changes.
They want to point the finger at Systems. We (systems) are not making
changes.
The 'DBA' belongs to the programmer group, and to be honest, I don't
think he has a clue concerning performance, just how to define the
databases.
At this point, I just need a better idea of where the problem really is
so I can attempt to throw it back to the DBA.
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
ITschak Mugzach
2018-08-02 18:53:03 UTC
Permalink
Clark gave you excellent advise to check what else is running at that time.
I once had a cics transaction that at 10 o'clock in the morning started
slow don. The DBA and his consultant claimed the system need more memory
and CPU. Omegamon told us a different story: We looked at the disk where
the table was located and discovered that a batch job reads the table
sequentially, so the disk heads was moving all the time, which cause the
slowdown. we shifted the job the problem disappeared saved us lot of money
in memory and cpu purchases ;-)

So my advise is dig! look at the CPU and elapse. may be your problem is
statistical.

ITschak
Post by Clark Morris
[Default] On 2 Aug 2018 08:43:32 -0700, in bit.listserv.ibm-main
Post by Tony Thigpen
1) There is no monitor.
2) WLM only has two profiles, DAY and NIGHT. These jobs all run during
the NIGHT settings.
3) Turf war prevents me from actually calling ADABAS support. Only the
DBA is allowed that option. And, if the issues are resolved with ADABAS
tuning, then he will look really bad as he has placed all the blame on
'Systems'. So, he will not call them for us.
I would compare all of the job statistics that you have access to
including CPU time and i-os to see what differs between normal and
spike nights. Also check what is running concurrently with the spike
night jobs. It may be a scheduling conflict. Also if Adabas is
cutting any SMF records that could be helpful get those and ask for
access to the tool(s) to interpret them.
Clark Morris
Post by Tony Thigpen
I need to get enough ammo to dump it back on the DBA.
Tony Thigpen
Post by Tony Thigpen
Is there anyone that would be willing to give me some guidance to solve
some ADABAS issues. Something is causing a performance problem and
nobody seems to even know where to start looking.
We have several Natural programs that run in minutes one night, then
take an hour on other nights. We also have some RPG programs (using the
VSAM bridge) that have similar problems.
The programmers don't think it has to do with daily work-load changes.
They want to point the finger at Systems. We (systems) are not making
changes.
The 'DBA' belongs to the programmer group, and to be honest, I don't
think he has a clue concerning performance, just how to define the
databases.
At this point, I just need a better idea of where the problem really is
so I can attempt to throw it back to the DBA.
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
--
ITschak Mugzach
*|** IronSphere Platform* *|* *Information Security Contiguous Monitoring
for Legacy **| *

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Carmen Vitullo
2018-08-02 18:55:42 UTC
Permalink
back in 2000 or so, MXG had some SAS process to analyze ADABAS PLOGS,



//ADALOG DD DISP=SHR,DSN=xx.ADABAS.PLOGCOPY
//SYSIN DD *
OPTIONS NOSOURCE NOFMTERR;
%INCLUDE SOURCLIB(XADALOG);
RUN;
been a long time since I've looked at ADABAS so if you have MXG, this 'may' help some


other than that, the SMF30's is the only other place I've looked

Carmen Vitullo

----- Original Message -----

From: "Clark Morris" <***@UNISERVE.COM>
To: IBM-***@LISTSERV.UA.EDU
Sent: Thursday, August 2, 2018 1:40:27 PM
Subject: Re: ADABAS tuning

[Default] On 2 Aug 2018 08:43:32 -0700, in bit.listserv.ibm-main
Post by Tony Thigpen
1) There is no monitor.
2) WLM only has two profiles, DAY and NIGHT. These jobs all run during
the NIGHT settings.
3) Turf war prevents me from actually calling ADABAS support. Only the
DBA is allowed that option. And, if the issues are resolved with ADABAS
tuning, then he will look really bad as he has placed all the blame on
'Systems'. So, he will not call them for us.
I would compare all of the job statistics that you have access to
including CPU time and i-os to see what differs between normal and
spike nights. Also check what is running concurrently with the spike
night jobs. It may be a scheduling conflict. Also if Adabas is
cutting any SMF records that could be helpful get those and ask for
access to the tool(s) to interpret them.

Clark Morris
Post by Tony Thigpen
I need to get enough ammo to dump it back on the DBA.
Tony Thigpen
Post by Tony Thigpen
Is there anyone that would be willing to give me some guidance to solve
some ADABAS issues. Something is causing a performance problem and
nobody seems to even know where to start looking.
We have several Natural programs that run in minutes one night, then
take an hour on other nights. We also have some RPG programs (using the
VSAM bridge) that have similar problems.
The programmers don't think it has to do with daily work-load changes.
They want to point the finger at Systems. We (systems) are not making
changes.
The 'DBA' belongs to the programmer group, and to be honest, I don't
think he has a clue concerning performance, just how to define the
databases.
At this point, I just need a better idea of where the problem really is
so I can attempt to throw it back to the DBA.
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN


----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Tom Marchant
2018-08-02 17:34:12 UTC
Permalink
Post by Tony Thigpen
1) There is no monitor.
No ADABAS monitor, or no system monitor?
Post by Tony Thigpen
3) Turf war prevents me from actually calling ADABAS support. Only the
DBA is allowed that option. And, if the issues are resolved with ADABAS
tuning, then he will look really bad as he has placed all the blame on
'Systems'. So, he will not call them for us.
I need to get enough ammo to dump it back on the DBA.
In that case, I would look at the ADABAS address space to determine
whether it is having trouble getting resources. Look for I/O statistics.
Document what you find. Give him the reports and tell him that ADABAS
seems to be performing well, as far as the system is concerned.

Ask him what makes him think that there is a system problem.
--
Tom Marchant

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Brian Westerman
2018-08-04 07:21:07 UTC
Permalink
Hi,

I have maintained many sites with Adabas and I have found common several factors tend to make it slow down. If it's fast sometimes and slow others, then it's almost always going to be a WLM problem.

You have to have a common hierarchy in the system, A site with Adabas is going to "normally" be one where adabas needs to be higher than almost everyone else including any other applications you are running, if you are running Com-Plete (SAG's tp monitor) it needs to be higher than TSO because TSO is a single person, Com-Plete can be hundreds or thousands. Think of Com-Plete as being like CICS. Putting a batch job that uses Adabas at a better position within WLM than Adabas is highly counter-productive.

One other big factor is the processor complex you are running on. Adabas likes a big fast CPU and while having multiple CPU's is helpful so that Adabas can monopolize one for itself, when it comes to Adabas, vertical capacity is much better than number of CPUs.

These numbers I'm about to mention are going to be relative to the rest of your system so keep in mind that 35 to one site might be like 95 at some other site, so just take them as general guidelines. When it comes to WLM, Adabas needs to be at a constant velocity setting, all in one period with an importance of '1'. For instance if Adabas is a velocity of 65, then Com-Plete would be 55. TSO should have a goal to complete with something like 80% in the first 0.6 seconds and then the second period you can go to a velocity of something low like 15. BATCH adabas jobs (those that use Natural and Adabas directly should be somewhere in the vicinity of a velocity of 25 for the first period with a duration of around 1000 with a second period of a straight velocity of something like 10. Don't put them in periods that swap out, and Adabas should never, ever, ever, ever be swapped out.

Most of the work is performed by the Adabas region on behalf of the batch jobs, with the batch jobs pretty much just doing the moving of field around that Adabs provides so they should not have a higher dispatiching priority or importance than Adabas itself, never, ever, ever.

As far as Adabas is concerned, NEVER put your Asso and DATA on the same volume, at least not for production. Assuming you are on a raid dasd box, it doesn't really matter where you place most things because they are all likely striped across the same physical volumes anyway, but Adabas does lots of things based on where he "thinks" things will be the most efficient.

Adabas needs a LOT of buffers for everything, if you think you have enough, you don't, until you see the shutdown stats and see how close you came to running out.

Make sure you have 64bit i/o turned on and that you are using large pages if possible. If you have not turned on large paging, turn it on and re-ipl your LAPR to make it available to Adabas, it's a noticeable difference. Adabas should be also ALWAYS be set to region=0M. Make sure your ECSA is VERY large, Adabas aware of how much you have and will throttle itself so as not to run out. It's all pageable, so make it big.

Depending on the version of Adabas, and how good (or not) that the installer was, you could be running with a lot of default parms as delivered by SAG, which are really great for initial testing but completely suck when it comes to day to day production response.

Also, in general, your tuning needs to be approached with some sort of goal in mind. Is Adabas access and performance the #1 priority? If not, it should be, because whether you run CICS or Com-plete as the tp monitor, if Adabas slows down, they will as well. Also you have to remember that your processor is a finite resource, you can only get so much out of it and when you hit 100.x% busy, something else has to give, and you don't want that to EVER be Adabas.

You have to make a list of everything you run and put it in order by importance to the site, and then remember that if something needs some other regions resources, then the "other" region is by definition more important. Everyone needs Adabas at most sites that run it, but Adabas doesn't really need VTAM or TCP, or BATCH, or TSO, or CICS or Com-PLETE, and having great VTAM or TCP throughput is silly when all of your users need Adabas to get anything done.

Programmers are the first to complain about this kind of thing, they tend to feel that they need to be higher than production, and there are even system programmers who feel that they have to have their TSO ID the highest priority in the system because they need to "get in" if something goes wrong. I hear that a lot and it still makes me laugh every time.

If you post your WLM settings here, we can all help you tune you system as an exercise. We probably won't all agree on things, but I would expect that your settings are poorly written right now and all help would probably be beneficial for you.

Just don't think you have to do all of this quickly, the process of going from bad to good is more iterative than you might first think. You have to make some guesses at first, and then monitor to see how it's going and react to that, so it might take you a week or so to get things going in the right direction, and even then you will still need to tweak things from time to time. There is always something that you didn't think about that comes up to hit you in the face, you just need to react to it and not overreact. Overreacting is counter-productive because it will mask true issues and normally exacerbates the problem you were trying to respond to in the first place

After all, tuning is really (unless you can order an upgrade) a balancing act to get the most out of your box. You have to have favorites, and you have to have things that you are willing to sacrifice support for in order to service your favorites.

If you want to do this off list, I'm okay with that as well, but I think it really would be better for you to get the opinions of a lot of different people on tuning. No one is completely right, and I would expect that (mostly) no one is completely wrong with their opinions of how things should be set up. Every site is different and has different priorities. Adabas sites and DB/2 sites will disagree about a lot of things, but generally the approach is the same.

A good first step would be to go to you WLM panels, print your current settings and post it here, or email it to me and I'll go over them with you. Tuning is something that everyone should know how to do, and unfortunately, not too many people do that any more. A lot of sites take the Watson&Walker quickstart samples and run them without any changes. It's a really, really good place to start, but it's just a start and a lot of people seem to miss that.

Brian

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to ***@listserv.ua.edu with the message: INFO IBM-MAIN
Loading...