Merric Reese

Warning to all – this is a very long & somewhat technical post…

You know, I’m really not a programmer any more, I can’t remember the last time I wrote a piece of production code – and in my current role – believe me – that’s probably a good thing – I now leave that to the expert Java developers in my team.  But, I came across my first MythTV recording that was from CH10 (Aus) and like all commercial channels it had ads…Argh!!!

To be honest – I don’t watch that much commercial TV anymore – so to be honest, this is rare, but it was a show that we regularly watch so – it became a priority

Anyway, from my recent experience, MythTV does a really great job of recording anything you point it at and it’s also very good at finding and flagging most of the TV ads but it’s really not very good at finding the start & end of a show – and trust me if you record anything on CH10 unless you leave 5 mins before the scheduled start of the show and at  least and 15 mins after the scheduled finish – you are going to miss it.  To magnify issues somewhat – we were watching the show upstairs on a non MythTV player – SO THERE WAS NO AUTO AD SKIPPING – and at that point – it was game over – I needed to find something to fix the ad problem or next time… (you know the rest)…

Well, as always in IT, the answer is always more complicated that was originally envisioned… So there are two parts to this simple endeavor 1) something that will trim the file while not messing with the video / audio sync and 2) something that will trim the start and end + delete the ads.  If you have ever dealt with TV Streams you will appreciate that fulfilling #1 is one of the toughest requirements – else you end up with video and audio that is not in sync.

The answer to part 1 (a/v sync) was something I found a little while ago – a program called — ProjectX – why? don’t know – but who every wrote the latest portable copy (2006) needs to be congratulated – It’s the best software ever for dealing with DVB-T streams, correcting any steam errors and generating files without video / audio sync problems.

Part 2 – well you’d think that would be MythTV… Yes, well ok… you would be mostly right – MythTV does a great job of finding ads on commercial channels & set the ads in and out points but if you want to take that and watch it on another player… well… it creates a NUV file (good luck trying to view this in anything else but MythTV) so to resolve my requirements – I ventured into the darker, inner workings of the mythconverg database and well — the rest is scriptory…

My goal- was to create a script that would generate files that could be opened directly in ProjectX to create a cutlist without any editing and to provide a viable cut list for further refinement (the input cutlist would be either based on parsing when the ads occurred or on a manual cutlist created using the MythTV editor).  After a few hours I was able to create the BASH script below (works for me – provided as is without warranty etc…

Ok the BASH script below is a proof of concept – the way it appears below assumes you are using mythrename for your vanity filenames, that you have this script available in your $PATH statement and that you are calling it from the same directory as the original vanity file.  BTW sorry about the formatting (wordpress seems to be removing the tabs) and for some reason the code tag doesn’t work with this theme.


#!/bin/bash

# Author: Merric Reese
# Date: 20100207 00:24
# Filename: projectx_cutlistsh
# Dependencies: None

##### Start Define Global Variables #####
mythuser=”<put your MythTV Database Username here>”
mythpass=”<put your MythTV Database Password here”
mythdb=”mythconverg”
chanid=”"
starttime=”"
in_filename=”"
in_fileext=”"
base_name=”"
out_filename=”"
out_fileext=”xcl”
frame_commlist=”"
frame_cutlist=”"
byte_offset=”"
byte_offset_commlist=”"
byte_offset_cutlist=”"

#echo args: $1

##### End Define Global Variables #####

##### Start Define Functions #####

list_all_variables() {
echo
echo — Listing all variables —
echo mythuser: $mythuser
echo mythpass: $mythpass
echo mythdb: $mythdb
echo chanid: $chanid
echo starttime: $starttime
echo in_filename: $in_filename
echo in_fileext: $in_fileext
echo base_name: $base_name
echo out_filename: $out_filename
echo out_fileext: $out_fileext
echo frame_commlist: $frame_commlist
echo frame_cutlist: $frame_cutlist
echo byte_offset: $byte_offset
echo byte_offset_commlist: $byte_offset_commlist
echo byte_offset_cutlist: $byte_offset_cutlist

}

#usage: get_basename
#returns: updtes base_name to the mythfilename with no preceeding directory info
get_basename_() {
base_name=”`basename $1`”
# in_filename=$base_name
in_filename=”`echo $base_neam | cut -d . -f 1`”
in_fileext=”`echo $base_neam | cut -d . -f 2`”
}

#usage: get_basename_vanity
#returns: updtes base_name to the mythfilename with no preceeding directory info based on using a vanity symbolic link filename generated with mythrename
get_basename_vanity() {
# echo getbase: $1
local lin_filename=”`basename $1`”
in_filename=”`echo $lin_filename | cut -d . -f 1`”
in_fileext=”`echo $lin_filename | cut -d . -f 2`”
local lfile=`ls -al $1 | cut -d \> -f 2`
base_name=`basename $lfile`
}

# ussge get_chanid_from_mythfilename
get_chanid_from_mythfilename() {
chanid=”`mysql -u$mythuser -p$mythpass –skip-column-names $mythdb -s -B –exec “select chanid from recorded where basename=’$1′;”`”

}

# ussge get_starttime_from_mythfilename
get_starttime_from_mythfilename() {
starttime=”`mysql -u$mythuser -p$mythpass –skip-column-names $mythdb -s -B –exec “select starttime from recorded where basename=’$1′;”`”

}

# get_frame_commlist()
# Dependencies: None – but can be used with get_chanid_from_mythfilename() and get_starttime_from_mythfilename() to feed chanid and starttime parameters
# Description: Gets the frame based commerical list cutlist from the recordedmarkup table in the mythconverg database based on a unique chanid and starttime – outputs a list of frames for a cutlist
# usage: get_frame_commlist
# eg: get_frame_commlist 1010 2010-02-04 18:55:00
get_frame_commlist () {
#echo inside commlist: $1 $2 $3
frame_commlist=`mysql -u$mythuser -p$mythpass –skip-column-names $mythdb -s -B –exec “select mark from recordedmarkup where chanid = $1 and starttime=’$2 $3′ and mark 0 and type in (4,5) order by mark asc;”`

}

# get_frame_cutlist()
# Dependencies: None – but can be used with get_chanid_from_mythfilename() and get_starttime_from_mythfilename() to feed chanid and starttime parameters
# Description: Gets the frame based cutlist from the recordedmarkup table in the mythconverg database based on a unique chanid and starttime – outputs a list of frames for a cutlist
# usage: get_frame_cutlist
# eg: get_frame_cutlist 1010 2010-02-04 18:55:00
get_frame_cutlist () {
#echo inside cutlist: $1 $2 $3
frame_cutlist=`mysql -u$mythuser -p$mythpass –skip-column-names $mythdb -s -B –exec “select mark from recordedmarkup where chanid = $1 and starttime=’$2 $3′ and mark 0 and type in (0,1) order by mark asc;”`

}

# get_byte_offset()
# Dependencies: None
# Description: Gets the closest byte offset for a specific mark from the mythconverg database based on a unique chanid, starttime and mark (note makes are ususally frames)
# usage: get_byte_offset
# eg: get_byte_offset 1010 2010-02-04 18:55:00 3765
get_byte_offset () {
# echo inside get_byte_offset: $1 $2 $3 $4
byte_offset=`mysql -u$mythuser -p$mythpass –skip-column-names $mythdb -s -B –exec “select offset from recordedseek rs where rs.chanid=$1 and rs.starttime=’$2 $3′ and rs.mark <= $4 order by offset desc limit 1;”`
}

# get_byte_offset_commlist()
# Dependencies: get_byte_offset(); $frame_commlist
# Description: Creates a list of closest byte offsets for a specific set of mark|frame from the mythconverg database based on a unique chanid, starttime and mark|frame that is provided to it from a file
# usage: get_byte_offset_list <
# eg: get_byte_offset_list.sh < markfile.txt 1010 2010-02-04 18:55:00
get_byte_offset_commlist() {
# echo inside get_byte_offset_list_commlist: $1 $2 $3 $4
local lfc1=”$frame_commlist”
local lfc2=”$frame_commlist”
# echo $lfc1
lfc1=`echo $lfc2 | cut -d ‘ ‘ -f 1`
# echo $lfc1 : $lfc2
local i=”1″
while [ "$lfc1" != "" ]
do
#echo inloop:$lfc1

#get & return the byte value
get_byte_offset $1 $2 $3 $lfc1
byte_offset_commlist=”$byte_offset_commlist $byte_offset”
#echo byte_offset_commlist: $byte_offset_commlist

#update the index and get the next value
i=$(( $i + 1 ))
lfc1=”`echo $lfc2 | cut -d ‘ ‘ -f $i`”
echo $lfc1
done
}

# get_byte_offset_list_cutlist()
# Dependencies: get_byte_offset(); $frame_cutlist
# Description: Creates a list of closest byte offsets for a specific set of mark|frame from the mythconverg database based on a unique chanid, starttime and mark|frame that is provided to it from a file
# usage: get_byte_offset_list <
# eg: get_byte_offset_list.sh $out_filename\_comm.$out_fileext
}

#write_file_header()
write_file_header_cut() {
echo CollectionPanel.CutMode=1 > $out_filename\_cut.$out_fileext
}

#write_byte_commlist()
#usage write_byt_commlist
write_byte_commlist() {
# echo inside write_byte_commlist: $1 $2 $3 $4

# write_file_header_comm

local lbc1=”$byte_offset_commlist”
local lbc2=”$byte_offset_commlist”
# echo $lbc1
lbc1=`echo $lbc2 | cut -d ‘ ‘ -f 1`
# echo $lbc1 : $lbc2
local i=”1″
while [ "$lbc1" != "" ]
do
#echo inloop:$lbc1

echo $lbc1 >> $out_filename\_comm.$out_fileext

#update the index and get the next value
i=$(( $i + 1 ))
lbc1=”`echo $lbc2 | cut -d ‘ ‘ -f $i`”
# echo $lbc1
done

}

#write_byte_cutlist()
#usage write_byt_cutlist
write_byte_cutlist() {
# echo inside write_byte_commlist: $1 $2 $3 $4

# write_file_header_cut

local lbc1=”$byte_offset_cutlist”
local lbc2=”$byte_offset_cutlist”
# echo $lbc1
lbc1=`echo $lbc2 | cut -d ‘ ‘ -f 1`
# echo $lbc1 : $lbc2
local i=”1″
while [ "$lbc1" != "" ]
do
#echo inloop:$lbc1

echo $lbc1 >> $out_filename\_cut.$out_fileext

#update the index and get the next value
i=$(( $i + 1 ))
lbc1=”`echo $lbc2 | cut -d ‘ ‘ -f $i`”
# echo $lbc1
done

}

##### End Define Functions #####

##### Start Program #####
echo calling function
get_basename_vanity $1
get_chanid_from_mythfilename $base_name
get_starttime_from_mythfilename $base_name
echo $chanid $starttime
get_frame_commlist $chanid $starttime
get_frame_cutlist $chanid $starttime

get_byte_offset $chanid $starttime 4880
get_byte_offset_commlist $chanid $starttime
get_byte_offset_cutlist $chanid $starttime

out_filename=$in_filename
write_file_header_comm
write_file_header_cut

echo $out_filename\_comm.$out_fileext
write_byte_commlist
write_byte_cutlist

list_all_variables

#—- end

— Ok take the above script is a beta (it works but I’m still looking for improvements)  – I’m sharing this with everyone as there seems to be some demand to link MythTV commercial flagging and cutlists to ProjectX (and some confusion about how to do this – hopefully the above script describes how to access the tables required to extract this information – please keep in mind that I’m still cleaning the above script up – I will work on incorporating any updates or changes suggested via the comments below in the script.

I also know that after reading some of this post – if your switched on – the alarm bells will start ringing in your head, the light bulbs will start going off (that’s how it happened for me)  and you will start to imagine how this process can be more efficient – please make sure you send me an email or post back to here so everyone can benefit from the illumination of your light bulb.

Future ambitions for this script:

• Improved comments to improve readability for myself and others.
• Allow command line parameters to tell the script to only produce one of the 2 types of files and switch between using the vanity filename or the MythTV Filename.
• Integrate / automate the “mythcommflag –gencutlist -f  <filename>” command.
• Re-factor the functions to remove the need for duplicate functions to generate both the commercial and cutlist files.

Please provide feedback as it will improve this beta script…

Thanks

Merric Reese