Translate

Saturday 27 January 2018

Raspbery Pi3 (RPI3) as a multimedia server and client, and general purpose PC  

  • DLNA/UPNP server with  local and remote (Internet) clients, DVR client and server, Subsonic server with Android client
  • Use of the Raspberry Pi as a general for web browsing and access to email using Dillo, Kweb, Netsurf, Yahoo! mail and Gmail   


These are some technical notes on:
  • streaming music and video services over a LAN and over the Internet using two Rasperry Pis
  • use of a Pi3 as a general purpose PC  for web browsing and email using Dillo, Netsurf, Kweb and Alpine (including some tips on using this software on the Pi)

These are not comprehensive notes but some tips on what can be time-consuming to set up initially but quite quick to install if a defined process is followed.

Hardware and software
  • RPi3 running 
    • Web browsers
    • Kodi (inc a mythtv client) and mythfrontend 
    • Subsonic server
    • minidlna (DLNA compliant server)
    • bubbleupnpserver available over the Internet
    • email clients 
    • libreoffice
  • Separate (remote on the LAN) RPi3 running:
    • mythbackend and connected to a PCTV USB Freeview tuner
  • Panasonic Viera TX-40DX700B as a DLNA client
  • Google Pixel as an Android client running:
    • Subsonic clients
    • mythtv client (using the excellent mythling app)

Startup
Some basic notes on setting up an RPI3 for multimedia 

Connect a minimum 8GB flash card to another PC.  For Linux users the flash card can be formatted with Gparted. Now mount the flashcard

sudo mount [flash card device]

Download NOOBS onto the flash card. For some reason, I find NOOBS a more reliable way of booting a new Pi than the Raspbian image. Unzip the zip file onto the flash card place into the pi and boot to the Pi. On an 8 Gb card there is not enough room for the full Raspian graphical desktop install so a workaround is to install Raspian lite and then upgrade to a graphical desktop, e.g.:

sudo apt-get update
sudo apt-get dist-upgrade
sudo apt-get install lxde [if you want lxde]

This takes up less space than the default graphical install.

The first two steps must be carried out otherwise the relevant packages will not be retrieved.  Next some configuration is required in order to make the display look acceptable and make some other useful tweaks:

sudo raspi-config

Some useful settings at this stage (which are well-documented elsewhere):

set the monitor resolution
set an overscan option (if black borders appear around your screen)
change the user password
allocate 256B to GPU memory for smooth HD video playback
set an audio output option (analog headphone socket or HDMI) 

Web browsers 

Standard Web browsers can be extremely slow on the Raspberry Pi. Some recommendations below.

dillo 

sudo apt-get install dillo

A very fast and lightweight browser best for searching and reading web pages which are predominantly text (pages with frames can be almost unusable). No Javascript so many web pages won’t have their full functionality but it can feel like a holiday not to be continually bombarded by adverts.

This browser has an interesting history as it originated with the noble aim of ensuring everyone could access the Internet no matter how basic their connection equipment (e.g. dial-up)

Tip. It is interesting to launch dillo from a terminal window and look in the terminal window at what dillo shows about which sites are trying to receive your browsing history. Example:

Domain: DENIED www.bbc.co.uk  b.scorecardresearch.com

kweb 

A good compromise between speed and functionality. Can be installed as follows (for Stretch):

wget http://steinerdatenbank.de/software/kweb-1.6.9.tar.gz
tar -xzf kweb-1.6.9.tar.gz
cd kweb-1.6.9
./debinstall

Installing from the git repsitory is more complicated because dependencies are not automatically installed

You can comfortably use kweb for a site like blogger which needs Javascript.

Tip. You may find it necessary to install the following additional package:

sudo apt-get install python-tk Tip.

Tip. Install omxplayer (a brilliantly efficient media player) so you can press the Play button in kweb and play media without Javascript or cookies enabled

Tip. Keep Javascript switched off for speed of browsing and Cookies off for privacy. Enable both if required for navigation/page rendering aannd certainly for Internet shopping. Complex sites may however make the Pi run very slowly.m

Tip. Many of us are used to entering searches in the browser address bar. Kweb can do this if the search string is prefixed by ? e.g.:

?how to install kweb

netsurf

Fast, and the page rendering is good but lacks Javascript, apparently not by design but because Javascript support has not yet been implemented.

Feels more like a standard browsing experience than dillo (although not as fast). Tabbed browsing is helpful

Using the Raspberry Pi for e-mail

For a lightweight client Claws Mail is a good choice. Can handle multiple email providers including  Yahoo! and Gmail.

For Web mail, Gmail works well in html view

Yahoo! Mail uses far too much memory in the default view. With Javascript switched on, go into settings and switch the view to Basic. You can subsequently access Yahoo mail without the Javascript and hence without the adverts.

There is a problem using Yahoo! Mail with Netsurf because Yahoo seems to think Netsurf is a mobile phone (and it seems impossible to configure the Netsurf User Agent in order to present Netsurf as another browser to Web sites). Hence you can't seem to get beyond a page promoting the Yahoo! app. A workaround is to display the Inbox in another browser and copy the mailbox URL into a Netsurf bookmark. In Netsurf you then need to log on (with cookies enabled) and then call up the Inbox from the bookmark.

Yahoo! Mail feels clunky in any view on the RPi. If you have multiple email providers a good option is to configure Gmail to use the other provider(s).  Note that the option to do this (Accounts and Import)  is not present in HTML view, you have to switch to standard view. However, once you switch back to HTML view you can see, and use, the imported emails. You can configure Gmail to respond to emails using the recipient's identity (e.g. respond to an incoming Yahoo! email using your Yahoo! email identity) and this works in HTML view. However in HTML view you can't seem to choose which identity to use at the point of replying (that appears to need Javascript).   

Alpine

Alpine is a fast and highly configurable text-based email client. There are a number of guides to setting up alpine with gmail including this one.

If you are using 2-step verification with gmail you need to set up a 16-character password associated with alpine. The command
touch .pine-passfile
in your home directory will cause Alpine to prompt you to save the password.


Alpine address book 

The Alpine address book is typically stored in ~/.addressbook and has a simple tab-delimited format. It is fairly straightforward to convert address books from other email clients into the Alpine address book format.  

Here is an example of such a conversion. If you can export and address book in the LDAP Data Interchange Format (LDIF) (e.g. in Claws-mail), there is a useful shell script you can download to convert an LDIF file to CSV. To use this script, download it, make it executable, "cat" the file and then pipe the output to the shell-script, adding to the shell script name a space-separated list of LDIF fields, e.g.

cat ldif.txt  | ./ldif-to-csv.sh givenName displayName mail > output.csv

where givenName etc. are the LDIF field names. 

To note that inverted commas must be edited out of the LDIF file otherwise the shell script will not work. 

The file can now be opened in Libreoffice calc, edited, and then output as a tab-delimited file (which should be saved in ~/.addressbook.
   


mythtv client


In my experience kodi works better as a mythtv client on the pi I assume because the excellent omxplayer is used (if you set it) to play the video. The mythtv client is good for SD resolution, HD resolution is OK but memory usage has to be minimised as far as possible. kodi playback is detectably better.

I tested this using a short section of recorded video where the mythtv frontend produced a small amount of pixellation with the mythtv frontend but no pixellation with the kodi frontend.    

the mythtv frontend

Instructions here: https://www.mythtv.org/wiki/MythTV_Light

As noted in the instructions, the firmware does need to be updated otherwise the frontend will report missing libraries and will not start. Don't try to skip the firmware update.

Memory dedicated to non-mythtv tasks needs to be minimised. The graphical desktop or windows client makes a big difference, I use dwm which is reported to run on only 1MB of RAM. A standard window manager can be abandoned altogether and just the mythtv frontend run instead. For instance a .desktop file can be created in /usr/share/xsessions such as the following:

[Desktop Entry]
Encoding=UTF-8
Name=myth
Comment=Mythtv frontend
Exec=mythfrontend
Icon=dwm # I retained this icon
Type=XSession

With this file created, an option to use mythfrontend appears in the menu on the password screen of Light Display Manager. So you run mythfrontend instead LXDE, Openbox etc. This is not officially endorsed because there can be problems running external programs from mythfrontend but for functionality internal to mythtv it seems to work fine.

In the video profile screen of mythfrontend, OpenMAX High Quality or openMAX normal must be used. In addition, select the Edit button against the "if" statement governing the selection of a renderer:

if res > [etc]

and make sure that the decoder is openmax.

For sound, use mythtv's internal volume controls and set the mixer as OpenMAX (analog or digital depending on your setup).

It is possible to create a directory of links to mythtv recordings using the Perl script mythlink.pl and run omxplayer from the command line. The script (which works fine with v.0.28 of mythtv and which resides in /usr/share/doc/mythtv-backend/contrib/user_jobs/) should be excutable. It needs an additional Perl module which can be installed as follows:

 sudo cpan install IO::Socket::INET6

The script can be run as simply as:

./mythlink.pl --link [outputdirectory]

The location of the database is automatically detected and the default options make identification of the relevant file straightforward.

kodi as a mythtv client

The performance of kodi as a mythtv client on the Pi is impressive, both the interface and the playback. I think this is the way to go for viewing mythtv content   To install:

sudo apt-get install kodi
sudo apt-get kodi-pvr-mythtv

There is little to configure if the backend is set up correctly. HD playback is crisp and smooth.

The command line may still be useful if you wanted to sequentially play all media in a directory for which the following can be used:

for file in [directory]/*
do
omxplayer $file
done

minidlna
  
Minidlna allows a DLNA-compliant server to be run on the Pi. It is simple to set up and works very well with my Panasonic Viera TX-40DX700B Smart TV as a client.

There is plenty of guidance on the Web. A key setting in the configuration file is:

inotify=yes

which means the database will be updated as soon as any new media is added to the relevant directories.    
 

Subsonic 

Download Java from the Oracle Website. As far as I know there is no way to install the right version of Java using apt-get.

This is a useful site for information about installing Java and setting up Java environment variables:

https://lifehacker.com/how-to-turn-a-raspberry-pi-into-a-private-streaming-mus-1583221462

A deb file can be downloaded from here [relfects the current version at the time of writing]:

http://www.subsonic.org/pages/download2.jsp?target=subsonic-6.1.3.deb

then:

sudo dpkg -i subsonic-6.1.3.deb

Note that, as downloaded, transcoding will not work because Subsonic installs incompatible binaries for ffmpeg and lame  in /var/subsonic/transcode. This has the effect that Subsonic will try to transcode a media file (if the original file is larger than the allowable streamed size) but this will fail and the Android client sticks at "Downloading 0B". Install ffmpeg and lame separately and copy to the Subsonic directory:

sudo apt-get install ffmpeg
sudo apt-get install lame 
sudo cp /usr/bin/ffmpeg /var/subsonic/transcode/ffmpeg 
sudo cp /usr/bin/ffmpeg /var/subsonic/transcode/lame

Subsonic is configured using a Web interface:

http://localhost:4040

I find this is never accessible by default if the Subsonic service is run as a user (rather than root). It appears to be something to do with permissions on the /var/subsonic folder (where the database resides). I find it is possible to get the service running as a user (rather than root) by changing ownership of /var/subsonic to the user. This seems to also require deleting the database in /var/subsonic/db and letting Subsonic recreate it. Due to a lack of documentation it is difficult to know exactly what fixes this but using these techniques I can get the service running as a user.    

bubbleupnpserver

Very useful and flexible server which acts as container for other UPNP servers and provides access to them over a LAN and over the Internet. All DLNA and UPNP server content can be made available including mythtv. Well documented and has extensive configuration options, especially on the mobile client.   

Instructions are here:

https://bubblesoftapps.com/bubbleupnpserver/

To install on the Pi:


wget https://launchpad.net/~bubbleguuum/+archive/bubbleupnpserver/+files/bubbleupnpserver_0.9-5_all.deb

sudo dpkg -i bubbleupnpserver_0.9-5_all.deb

Make sure port 58050 is open as the service is configured using:

http://localhost:58050

I had no problems connecting (unlike Subsonic). 

Connection over the Internet is possible using a dynamic DNS service.

The version of ffmpeg installed using apt-get must be copied to the /usr/share/bubbleupnpserver folder on the server otherwise transcoding will not work.  The default binary will not work on the Pi (same issue as Subsonic). 


It is straightforward to download music and videos at different levels of the folder hierarchy and then play the media offline. So a directory, album, song etc can be downloaded. 

Android clients

mythling

A reliable mythtv client. Provides, on both the local network and the Internet access to:
  • Live TV
  • Recordings
  • Programme guide
  • Videos
  • Music  
In order to access the Mythtv over the Internet you just configure Mythling with your external address, preferably using a static IP service (dynDNS, No-IP etc) if your external address is dynamic and likely to change frequently.