I am not a big fan of syntax highlighting - I fail to see the use for anything other than judging the length of a string or comment.
Based on the Darcula theme, I uploaded a color scheme that disables all colors other than the above.
drivel, my Twitter CLI, has some new capabilities now. I moved the original functionality to the ‘status’ subcommand, and added subcommands for home, mentions, replies and likes.
For the fun of it, I decided to reimplement a basic version of pm5conv in LaymansHex and a bit of Bash - to parse workout data of a Concept2 PM5 monitor.
I basically took the data format description and translated it to LaymansHex definitions and Bash glue:
% for i in `ls | grep -E "(layhex|bash)"`; do echo "==> $i\n"; cat $i; echo; done
==> LogDataAccessTbl.bin.layhex
little endian
: byte[offset*32]
Magic : byte[1]
WorkoutType : uint8
: byte[10]
NoSplits : uint16
: byte[2]
Offset : uint16
: byte[6]
Size : uint16
Index : uint16
: byte[4]
Marker : byte[32]
==> LogDataStorage.bin-Workout1.layhex
big endian
: byte[offset]
: byte[1]
: byte[1]
: byte[2]
: byte[4]
Timestamp : byte[4]
UserID : uint16
: byte[4]
RecordID : uint8
: byte[3]
TotalDuration : uint16
TotalDistance : uint32
SPM : uint8
: byte[1]
SplitSize : uint16
: byte[18]
: byte[splitNo*32]
SplitDistance : uint16
SplitHeartRate : uint8
SplitSPM : uint8
: byte[28]
==> pm5conv.bash
#!/bin/bash
cmd="laymanshex"
accTable="LogDataAccessTbl.bin"
storage="LogDataStorage.bin"
# uses evil eval
function printNonEmpty {
varname='$'"$1"
eval "val=$varname"
if [ "$val" != "" ]; then
echo "$1=$val"
fi
}
function printTimestamp {
timestamp="0x$1"
year="$(( ((timestamp & 0xFE000000) >> 25) + 2000 ))"
day="$(( (timestamp & 0x01F00000) >> 20 ))"
month="$(( (timestamp & 0x000F0000) >> 16 ))"
hour="$(( (timestamp & 0x0000FF00) >> 8 ))"
minute="$(( timestamp & 0x000000FF ))"
printf "Date=%d-%02d-%02d %02d:%02d\n" $year $month $day $hour $minute
}
function printSplit {
outputSplit="$($cmd -nopadding -fvar=offset=$1,splitNo=$2 $storage-Workout$3.layhex $storage 2> /dev/null)"
status=$?
if [ $status -eq 0 ]; then
eval "$outputSplit"
echo
echo "Split $(($2+1))"
echo "----------"
printNonEmpty "SplitDuration"
printNonEmpty "SplitDistance"
printNonEmpty "SplitHeartRate"
printNonEmpty "SplitSPM"
fi
}
function printWorkout {
echo "Workout $1"
echo "============="
outputHeader="$($cmd -nopadding -fvar=offset=$2,splitNo=0 $storage-Workout$3.layhex $storage 2> /dev/null)"
status=$?
if [ $status -eq 0 ]; then
eval "$outputHeader"
printTimestamp "$Timestamp"
printNonEmpty "TotalDuration"
printNonEmpty "TotalDistance"
printNonEmpty "SplitSize"
printNonEmpty "SPM"
j=0
while [ "$j" -lt "$NoSplits" ]; do
printSplit $2 $j $3
j=$((j+1))
done
echo
fi
}
function printAll {
i=0
while : ; do
offset=$(($i))
output="$($cmd -nopadding -fvar=offset=$offset $accTable.layhex $accTable 2>/dev/null)"
status=$?
if [ $status -ne 0 ]; then
break
else
eval "$output"
printWorkout $Index $Offset $WorkoutType
fi
i=$((i+1))
done
}
printAll
Output of pm5conv.bash:
% ./pm5conv.bash
Workout 1
=============
Date=2018-02-27 21:00
TotalDuration=962
TotalDistance=363
SplitSize=3000
SPM=29
Split 1
----------
SplitDistance=363
SplitHeartRate=0
SplitSPM=27
Workout 2
=============
Date=2018-02-28 20:41
TotalDuration=4208
TotalDistance=1559
SplitSize=3000
SPM=27
Split 1
----------
SplitDistance=1121
SplitHeartRate=0
SplitSPM=29
Split 2
----------
SplitDistance=438
SplitHeartRate=0
SplitSPM=26
Workout 3
=============
Date=2018-03-01 18:47
TotalDuration=12058
TotalDistance=4166
SplitSize=3000
SPM=28
Split 1
----------
SplitDistance=1056
SplitHeartRate=0
SplitSPM=29
Split 2
----------
SplitDistance=1060
SplitHeartRate=0
SplitSPM=28
Split 3
----------
SplitDistance=1007
SplitHeartRate=0
SplitSPM=28
Split 4
----------
SplitDistance=1024
SplitHeartRate=0
SplitSPM=29
Split 5
----------
SplitDistance=19
SplitHeartRate=0
SplitSPM=0
[...]
I have spent some time thinking about handling sequences and in-file offsets with LaymansHex. I decided on two things:
Both points led me to believe, that it’s better to handle sequences and offsets with multiple calls to LaymansHex and variable byte field sizes in the format definition to allow for offsets and sliding windows. I implemented this in commit 279429d0bff0681d533b23c260249361078d0a78
I added a minimal example to the of the README.
theHunter: Call of the Wild is a hunting simulator, which - unfortunately - has some characteristics of a typical first-person shooter, namely missions, rewards and locked equipment. Since this prevents actual hunting, I thought it would be interesting to analyse the file format that saves level, XP, cash and so on.
I wrote a fairly generic tool called laymanshex that takes a partial file description and a binary file as input and outputs the values. Values can be changed with the ‘-set’ argument.
For example:
% ./laymanshex thp_player_profile_adf.layhex thp_player_profile_adf
Level = 19
XP = 20235
SkillPoints = 0
PerkPoints = 0
SkillPointsSpent = 7
PerkPointsSpent = 6
Cash = 16640
RifleLevel = 14
HandgunLevel = 4
ShotgunLevel = 4
BowLevel = 1
RifleScore = 1992
HandgunScore = 382
ShotgunScore = 417
BowScore = 0
% ./laymanshex -set="Level=60,XP=100000" thp_player_profile_adf.layhex thp_player_profile_adf
Created backup thp_player_profile_adf.bak20200426231811
Level = 60
XP = 100000
SkillPoints = 0
PerkPoints = 0
SkillPointsSpent = 7
PerkPointsSpent = 6
Cash = 16640
RifleLevel = 14
HandgunLevel = 4
ShotgunLevel = 4
BowLevel = 1
RifleScore = 1992
HandgunScore = 382
ShotgunScore = 417
BowScore = 0
% ./laymanshex -set="Cash=0x7FFFFFFF" thp_player_profile_adf.layhex thp_player_profile_adf | grep Cash
Cash = 2147483647
The partial file description for thp_player_profile_adf (COTW version 1.49) looks like this:
little endian
: byte[113]
Level : int32
XP : int32
SkillPoints : int32
PerkPoints : int32
SkillPointsSpent : int32
: byte[60]
PerkPointsSpent : int32
: byte[60]
Cash : int32
RifleLevel : int32
HandgunLevel : int32
ShotgunLevel : int32
BowLevel : int32
RifleScore : int32
HandgunScore : int32
ShotgunScore : int32
BowScore : int32
Releases of laymanshex can be found here, go sources in the laymanshex git repo and possible further file descriptions in the laymanshex-files git repo.
The file format of COTW saves has changed many times in the past and I assume it will again at some point in the future.