DevOps & Modernization Troubleshooting
Monitoring alert lacks actionable context
Monitoring alert lacks actionable context means the pipeline moved the wrong bits, or moved the right bits to the wrong place. Diff what was deployed against what was intended.
What it means
- A mainframe deploy has three artifacts that must agree: the source, the compiler options, and the load module actually in the execution library.
- Copybooks are shared dependencies - one changed copybook silently changes every program that includes it.
- Working in test only proves the test artifacts agree, not that production does.
Common causes
- Stale load module: the link-edit ran against an older object instead of the fresh compile.
- Copybook changed in one library while an older copy remains in the compile concatenation.
- Wrong compiler options, for example a test-only option, baked into the production module.
- Promotion package built from the wrong source version or promoted to the wrong environment.
- Link-edit resolving a routine from an earlier library in the concatenation instead of the new one.
- Rollback restoring the load module but leaving new configuration or a new copybook layout in place.
How to diagnose
- Compare timestamps: source member, object, and load module - the load module must be the newest.
- Check the compiler listing to see exactly which library each copybook resolved from.
- Review the binder map for where each external reference resolved.
- In CICS use CEMT INQUIRE PROGRAM to confirm the loaded module matches what was promoted, then NEWCOPY or PHASEIN.
- Diff the test and production library concatenations member by member.
How to fix it
- Rebuild from the correct source with production compiler options and promote the fresh load module.
- Sync copybooks across libraries before compiling, and recompile every dependent program.
- Fix the library concatenation order so the intended library resolves first.
- After deploying to CICS, NEWCOPY the program and verify with CEMT INQUIRE.
- Roll back code and configuration together, and verify both afterwards.
Example: Rebuild with a listing that proves what was used
//BLD EXEC IGYWCL,PARM.COBOL='LIST,MAP',
// PARM.LKED='LIST,XREF'
//COBOL.SYSIN DD DSN=PROD.SOURCE(MYPROG),DISP=SHR
//COBOL.SYSLIB DD DSN=PROD.COPYLIB,DISP=SHR
//LKED.SYSLMOD DD DSN=PROD.LOADLIB(MYPROG),DISP=SHR
The LIST output proves which copybook versions the compile used; the XREF shows where each external reference resolved.
Interview tip
- Q: Production abends but the same load module works in test, and the source matches. Monitoring alert lacks actionable context - what do you suspect?
- A: If source and load module agree, the difference is in what surrounds them: the copybook used at compile time, the compiler options, or the library concatenation at runtime. I compare the production compile listing with the test one and check where the link-edit resolved each external. Most times it is a stale copybook or a concatenation order difference.
Prevention checklist
- Build promotion packages from version-controlled source with recorded compiler options, never by hand.
- Recompile all dependents whenever a shared copybook changes.
- Verify CICS NEWCOPY or PHASEIN completed after every online deploy.
- Roll back configuration together with code, and verify both.
