blog

Categories     Timeline     RSS

IDEA / Android Studio Theme

I am not a big fan of syntax highlighting - I fail to see the use for anything other than judging the length of a string or comment.

Based on the Darcula theme, I uploaded a color scheme that disables all colors other than the above.

drivel update

drivel, my Twitter CLI, has some new capabilities now. I moved the original functionality to the ‘status’ subcommand, and added subcommands for home, mentions, replies and likes.

Concept2 PM5 with LaymansHex

For the fun of it, I decided to reimplement a basic version of pm5conv in LaymansHex and a bit of Bash - to parse workout data of a Concept2 PM5 monitor.

I basically took the data format description and translated it to LaymansHex definitions and Bash glue:

% for i in `ls | grep -E "(layhex|bash)"`; do echo "==> $i\n"; cat $i; echo; done
==> LogDataAccessTbl.bin.layhex

little endian
            : byte[offset*32]
Magic       : byte[1]
WorkoutType : uint8
            : byte[10]
NoSplits    : uint16
            : byte[2]
Offset      : uint16
            : byte[6]
Size        : uint16
Index       : uint16
            : byte[4]
Marker      : byte[32]

==> LogDataStorage.bin-Workout1.layhex

big endian
                    : byte[offset]
                    : byte[1]
                    : byte[1]
                    : byte[2]
                    : byte[4]
Timestamp           : byte[4]
UserID              : uint16
                    : byte[4]
RecordID            : uint8
                    : byte[3]
TotalDuration       : uint16
TotalDistance       : uint32
SPM                 : uint8
                    : byte[1]
SplitSize           : uint16
                    : byte[18]
                    : byte[splitNo*32]
SplitDistance       : uint16
SplitHeartRate      : uint8
SplitSPM            : uint8
                    : byte[28]

==> pm5conv.bash

#!/bin/bash

cmd="laymanshex"

accTable="LogDataAccessTbl.bin"
storage="LogDataStorage.bin"

# uses evil eval

function printNonEmpty {
  varname='$'"$1"
  eval "val=$varname"
  if [ "$val" != "" ]; then
      echo "$1=$val"
  fi
}

function printTimestamp {
  timestamp="0x$1"
  year="$(( ((timestamp & 0xFE000000) >> 25) + 2000 ))"
  day="$((   (timestamp & 0x01F00000) >> 20 ))"
  month="$(( (timestamp & 0x000F0000) >> 16 ))"
  hour="$((  (timestamp & 0x0000FF00) >> 8  ))"
  minute="$(( timestamp & 0x000000FF ))"
  printf "Date=%d-%02d-%02d %02d:%02d\n" $year $month $day $hour $minute
}

function printSplit {
  outputSplit="$($cmd -nopadding -fvar=offset=$1,splitNo=$2 $storage-Workout$3.layhex $storage 2> /dev/null)"
  status=$?
  if [ $status -eq 0 ]; then
    eval "$outputSplit"
    echo
    echo "Split $(($2+1))"
    echo "----------"
    printNonEmpty "SplitDuration"
    printNonEmpty "SplitDistance"
    printNonEmpty "SplitHeartRate"
    printNonEmpty "SplitSPM"
  fi
}

function printWorkout {
  echo "Workout $1"
  echo "============="
  outputHeader="$($cmd -nopadding -fvar=offset=$2,splitNo=0 $storage-Workout$3.layhex $storage 2> /dev/null)"
  status=$?
  if [ $status -eq 0 ]; then
    eval "$outputHeader"
    printTimestamp "$Timestamp"
    printNonEmpty "TotalDuration"
    printNonEmpty "TotalDistance"
    printNonEmpty "SplitSize"
    printNonEmpty "SPM"
    j=0
    while [ "$j" -lt "$NoSplits" ]; do
      printSplit $2 $j $3
      j=$((j+1))
    done
    echo
  fi
}

function printAll {
  i=0
  while : ; do
    offset=$(($i))
    output="$($cmd -nopadding -fvar=offset=$offset $accTable.layhex $accTable 2>/dev/null)"
    status=$?    
    if [ $status -ne 0 ]; then
      break
    else
      eval "$output"    
      printWorkout $Index $Offset $WorkoutType
    fi
    i=$((i+1))
  done
}

printAll

Output of pm5conv.bash:

% ./pm5conv.bash
Workout 1
=============
Date=2018-02-27 21:00
TotalDuration=962
TotalDistance=363
SplitSize=3000
SPM=29

Split 1
----------
SplitDistance=363
SplitHeartRate=0
SplitSPM=27

Workout 2
=============
Date=2018-02-28 20:41
TotalDuration=4208
TotalDistance=1559
SplitSize=3000
SPM=27

Split 1
----------
SplitDistance=1121
SplitHeartRate=0
SplitSPM=29

Split 2
----------
SplitDistance=438
SplitHeartRate=0
SplitSPM=26

Workout 3
=============
Date=2018-03-01 18:47
TotalDuration=12058
TotalDistance=4166
SplitSize=3000
SPM=28

Split 1
----------
SplitDistance=1056
SplitHeartRate=0
SplitSPM=29

Split 2
----------
SplitDistance=1060
SplitHeartRate=0
SplitSPM=28

Split 3
----------
SplitDistance=1007
SplitHeartRate=0
SplitSPM=28

Split 4
----------
SplitDistance=1024
SplitHeartRate=0
SplitSPM=29

Split 5
----------
SplitDistance=19
SplitHeartRate=0
SplitSPM=0

[...]

Sequences in LaymansHex

I have spent some time thinking about handling sequences and in-file offsets with LaymansHex. I decided on two things:

  1. LaymansHex is not and should not be used as a “proper” binary parser. It is a quick-and-dirty tool to read and set some values.
  2. Binary sequences are implemented in multiple ways and adding every way to the file description format is tedious and makes the use complex.

Both points led me to believe, that it’s better to handle sequences and offsets with multiple calls to LaymansHex and variable byte field sizes in the format definition to allow for offsets and sliding windows. I implemented this in commit 279429d0bff0681d533b23c260249361078d0a78

I added a minimal example to the of the README.

theHunter COTW player file format

theHunter: Call of the Wild is a hunting simulator, which - unfortunately - has some characteristics of a typical first-person shooter, namely missions, rewards and locked equipment. Since this prevents actual hunting, I thought it would be interesting to analyse the file format that saves level, XP, cash and so on.

I wrote a fairly generic tool called laymanshex that takes a partial file description and a binary file as input and outputs the values. Values can be changed with the ‘-set’ argument.

For example:

% ./laymanshex thp_player_profile_adf.layhex thp_player_profile_adf
           Level = 19
              XP = 20235
     SkillPoints = 0
      PerkPoints = 0
SkillPointsSpent = 7
 PerkPointsSpent = 6
            Cash = 16640
      RifleLevel = 14
    HandgunLevel = 4
    ShotgunLevel = 4
        BowLevel = 1
      RifleScore = 1992
    HandgunScore = 382
    ShotgunScore = 417
        BowScore = 0

% ./laymanshex -set="Level=60,XP=100000" thp_player_profile_adf.layhex thp_player_profile_adf
Created backup thp_player_profile_adf.bak20200426231811

           Level = 60
              XP = 100000
     SkillPoints = 0
      PerkPoints = 0
SkillPointsSpent = 7
 PerkPointsSpent = 6
            Cash = 16640
      RifleLevel = 14
    HandgunLevel = 4
    ShotgunLevel = 4
        BowLevel = 1
      RifleScore = 1992
    HandgunScore = 382
    ShotgunScore = 417
        BowScore = 0

% ./laymanshex -set="Cash=0x7FFFFFFF" thp_player_profile_adf.layhex thp_player_profile_adf | grep Cash
            Cash = 2147483647

The partial file description for thp_player_profile_adf (COTW version 1.49) looks like this:

little endian
                 : byte[113]
Level            : int32
XP               : int32
SkillPoints      : int32
PerkPoints       : int32
SkillPointsSpent : int32
                 : byte[60]
PerkPointsSpent  : int32
                 : byte[60]
Cash             : int32
RifleLevel       : int32
HandgunLevel     : int32
ShotgunLevel     : int32
BowLevel         : int32
RifleScore       : int32
HandgunScore     : int32
ShotgunScore     : int32
BowScore         : int32

Releases of laymanshex can be found here, go sources in the laymanshex git repo and possible further file descriptions in the laymanshex-files git repo.

The file format of COTW saves has changed many times in the past and I assume it will again at some point in the future.

< Older

Newer >