Support viewing of VB binary records in HEX

[hammyau]

Zowe Explorer does not support viewing of binary files. Unless I am missing something.

I would like to be able to see my record based files and view them in hex.
Like ISPF browse or view. And switching to hex mode.

I can simply download a data set via FTP and use Vi or Notepad ++ or VS code. And its hex viewer.
But none of these understand the record format.

I believe I can transfer them via the Zowe CLI. But I have not tried. To me that would just be the same as using FTP.

I realize this will be quite an effort? I have looked through some of the code. And not quite sure how it would fit in.

I like Zowe Explorer, but without this feature it will be of limited use to my colleagues and I.

[dkelosky]

hi @hammyau - Can you share an example? You mentioned binary files but also mentioned needed to understanding the record format. What RECFM is used for these files on z/OS?

You can download binary files with zowe CLI, e.g. zowe files download ds "IBMUSER.LOADLIB" --file iefbr14.exe --binary. It would be and identical result to downloading with FTP. In either case, you can open this in VS Code with a hex dump extension (as you said):

However, the rendered characters will no be EBCDIC (the same in ISPF).

[FALLAI-Denis]

I am not sure that editing data files in Zowe Explorer is a good idea, for the following reasons:

• the data volume can be very large
• files containing data in a non-character format is problematic (some data can be converted from Ebcdic to ASCII / UTF8, but others cannot)
• working on files with variable length records can be problematic (keeping spaces at the end of the line)
• some file organization can be problematic (VSAM KSDS / RRDS)
• access to potentially sensitive data (confidentiality, security)

For the browse, even the edit, of data files, I think that Zowe Explorer should cooperate with an agent on the host:

• the data remains physically on the host
• Zowe Explorer uses a view on this data, with a positioning and pagination mechanism
• the modification of values ​​is done in the view
• a mechanism must allow switching from a "character" display to a "dump" display
• search functions (like FIND an ISPF EDIT)
• replacement functions (like CHANGE in ISPF Edit)
• support for COPYBOOK (COBOL), INCLUDE (PL/I), COPY (HLASM) for formatting data in browse / edit view

This agent on the host could be hosted in Zowe Server (which would give Zowe Server an interest, because with the use of VS Code and the various extensions, I can't find any reason to deploy Zowe Server).

[hammyau]

Thanks Dennis, I do think it will be a big task.
For many reasons.
Maybe editing binary records will just have to be an ISPF thing?
Even just being able to view them and their record based content in hex would be very helpful.

[fbeyl]

I agree that for full edit functionality it is not a good idea to do it with Zowe explorer but do it with ISPF isn't either and that for the exact same reasons.
For this purpose several vendors have products in the market (IBM File Manager, Broadcom CA File Master Plus, Compuware File Aid, ...) it would be good if they created rest API's and used Zowe Mediation Layer for these products.
But I think the original request was not for full functional editing but merely a possibility to download without conversion and browse / basic edit eventually in hex mode. I think this might be possible without agent on the host and not that much effort in VSC extension Zowe Explorer.
I will upload some screenshots and try to make my point in a next comment.

[fbeyl]

On the Tutorial system I created a variable blocked sequential dataset.

Records contain packed decimal, zoned decimal signed numeric fields as well as text fields. ISPF browse and edit use the recordlength and edit inform that the data contains non-display characters.

When setting hex on these are visible
.
In my VSC Zowe Explorer I can't download datasets in binary but I can for uss files. So I used postman to binary download from z/OSMF dataset API. However even when downloaded binary there are some issues. Data is no longer split in records and recordlength is not available. For a fb records the datasets lrecl can be used but for vb the lrecl is just the max length. However z/OSMF dataset API header X-IBM-Data-Type can be set to value record and this might just be what is needed. I redownloaded with postman and then still no record split but all records are preceded by the length in a 4 byte big endian.

[fbeyl]

So downloading dataset using 'record' instead of 'binary' and creating a hex viewer/editor that uses the preceding number of bytes following to determine records boundaries and translates the right codepage ( known from profile ) might do the job.

[hammyau]

Yes we will need the record lengths. As part of our project I have written code that parses the record layouts of our VB data sets. These are processed after FTP'ing them to my PC. On the PC the key is to FTP with the -RDW option to preserve the record descriptor word. Then I basically read 2 bytes to get the record length, read the remaining bytes for that record and repeat.

Where in the chain of events this happens between requesting to browse/edit a binary file via Zowe explorer and it being displayed I'm not sure. Having VSCode do the work may be too much for a large file. Unless we do something clever with the window of what is being displayed.

I can write a program that will do the conversion work for display. And then just have the VSCode display it. It could even edit it and I can have my converter write it back again. But that will all be separate from Zowe. I would have though it could be integrated in some way.

[fbeyl]

z/OSMF has a header to limit record retrieval ( X-IBM-Record-Range ), even has header to obtain enq. Zowe explorer already uses z/OSMF to retrieve the dataset so I suppose it shouldn't be to hard to fully exploit these headers.
Exploit them combined with hex edit taking into account charset/codepage and 'record' notion could be an important enhancement for Zowe explorer.

[fbeyl]

For large datasets the headers might be handy to limit bandwith. z/OSMF adds continuously new functionality ... headers to search, filter, return a range of records, ... Someone with Zowe Explorer development skills should have a look at them to discover new possibilities. I think Zowe Explorer should definitly follow this evolution.
Of course this way of browse/edit datasets should be used for developer/sysprog usage only.

[hammyau]

An interesting workaround for this was supplied by a colleague of mine.
Use IDCAMS to convert the file to hex on the mainframe and then use Zowe to examine the result.

This works as a way of examining binary files in hex. But editing them. That will still have to fall back to ISPF.

Attached some example JCL.
hex.txt

[FALLAI-Denis]

@hammyau @fbeyl
See if #1589 and #1590 can be a solution for this issue, (if ok, please vote).

To process VB files, you have to work in "record mode", and when uploading it is Zowe CLI to determine the length of each record.

[JillieBeanSim]

This enhancement has had no community activity for 12 months. The issue also has less than 10 up-votes by the community. No action on this enhancement is targeted for the next 2 calendar quarters. Therefore, this enhancement is being closed. If you feel that this enhancement should continue to be available for community up-votes, you may reopen this issue.