macOS shell script dd: write hex data from a variable

[Macschrauber]

Doing a shell script like:

myvar=$(dd if="$1" bs=1 skip=10 count=16)

printf $myvar >test.file

is the thing I want to do, but of course printf skips 0x00 and other none printable bytes.

right now I write a lot of temp files to parse things but I like to put the data into variables to avoid all those temp files.

thanx a bunch enlightening me

 

[Slartibart]

you could use for example gawk or od…

 

[mfram]

Check out the man page for the 'xxd' command. It does hexdumps and goes back and forth. It might be what you're looking for. You can have it write bytes to specified offsets inside a file.

 

[Macschrauber]

Thank you, guys, will look into those suggestions.

Tho I hoped for a construction what sets myvar to being the source for writing a binary file like it was a string.

Like cat $myvar $mavarb $myvarc>file.bin

 

[Slartibart]

use xxd as recommended by @Macschrauber, which seems the simplest way: concatenate the variables and pipe them to xxd (and redirect the output to a file).
There are plenty of additional options (sed, …) one can think of… but if you writing content to variables, why not write directly to a file in the first place?

 

[chown33]

Thank you, guys, will look into those suggestions.

Tho I hoped for a construction what sets myvar to being the source for writing a binary file like it was a string.

Like cat $myvar $mavarb $myvarc>file.bin

This doesn't concatenate the named variables. It concatenates the contents of the files whose names are given by the variables. This happens because 'cat' works with filenames, not literal strings.

If the value of a variable is wanted, you can use 'echo' or 'printf'.

If the value of a variable is a hexadecimal string, you can pipe the output of echo or printf to the 'xxd' command, and then redirect its output to a file.

Here's a contrived example:

varA="41424344"
varB="2032303233"
NL="0A"

# Works with or without spaces between hex sequences.
echo "$varA$NL$varB$NL" | xxd -r -ps  >test.txt
echo "$varA $NL $varB $NL" | xxd -r -ps  >>test.txt

# Dump the file in hex, with line labels.
xxd test.txt

 

[Macschrauber]

use xxd as recommended by @Macschrauber, which seems the simplest way: concatenate the variables and pipe them to xxd (and redirect the output to a file).
There are plenty of additional options (sed, …) one can think of… but if you writing content to variables, why not write directly to a file in the first place?

I have to modify the file contents, not just parsing them.

The construct is way more complex, I know about xxd and letting xxd write a binary file.

I hoped there is something I have overlooked. Am not too much experienced in shell scripting. But no absolute beginner.

I need to accept that I cannot use a variable's binary content directly as a >if source< for >dd<.

 

[chown33]

'dd' can read from stdin. 'xxd' can write to stdout. That means you can connect them in a pipeline.

Also, read the full man page for 'xxd'. The '-s' option lets you seek to a specific offset before writing, so like the 'skip' option to 'dd', you should be able use 'xxd' to directly write bytes at any offset within an output file.

 

[mfram]

Can you just write directly from perl or python or whatever scripting language you want to use? Tell the language to open the file in binary mode. Shell is a bit more... limited... on what it can do easily. Use a more powerful scripting language.

 

[Macschrauber]

ok, guys, I will dig a little deeper in xxd

 

[Macschrauber]

ok, I guess I can work with this (always remembering I work on a hexdump not binary)

odd: xxd adds a space character after 30 bytes into the output stream, so that tr -d ' '

#!/bin/sh
file_to_hexvar()
{
xxd -p "$1"
}

hexvar_to_file()
{
  echo $1 | tr -d ' ' | xxd -r -p >"$2"
}

myvar=$(file_to_hexvar $1)
hexvar_to_file "$myvar" $1.modded

xxd "$1"
xxd "$1.modded"

diff "$1" "$1.modded"