On my server running Ubuntu 12.04, I have Apache2 listening on port 8008 (20 virtual hosts) and Varnish on port 80. I use munin and monit to keep tabs on the various services running on the machine and they use the mod_status output to keep tabs on Apache2 processes. As mod_status is compiled into Apache2 by default and the module is enabled this means that the detailed information about your secure webserver is exposed for all the world to see. The mod_status configuation file allows you to lock access down to localhost access only which is nice.
However, Varnish connects to the backend Apache2 server as localhost and so exposes the server-status page to the wild wild web. With the Varnish VCL I was using my server-status page was cached for an hour so it was still available but only provided a static status report that changed when the cache refreshed. Implementing a solution from serverfault.com made the live server-status available through Varnish as it passed it to the backend. This did not help. To secure my server-status I had to block access to the server-status URL but still keep it open for munin-node. Continue reading
I have a software RAID1 (mirrored) setup with four partitions on two 160GB SATA drives. Munin started emailing me reports that a drive was failing with bad sectors.
This is what I did to get the system up and running again.
From advosys.ca I completed the following steps first in the terminal:
To make Ubuntu Server automatically boot when one drive in a RAID array has failed do the following:
From a running server, do a package update to make sure you have the latest kernel and boot loader
sudo apt-get update && apt-get upgrade
Reboot the server to ensure any new kernel and bootloader packages are in place.
From the command line run
sudo grub-install /dev/md0
to ensure GRUB is installed on all members of the boot RAID device.
When asked “Should mdadm run monthly redundancy checks of the RAID arrays?”, select either Yes or No (read the warning about possible performance impact and decide. “Yes” is the safer choice)
From the command line run
sudo dpkg-reconfigure mdadm
When asked “Do you want to start the md monitoring daemon?” select Yes.
Enter a valid email address to send warning messages to.
When asked “Do you want to boot your system if your RAID becomes degraded?” select Yes. Continue reading
The process in the previous post loaded one layer at a time. This batches the process.
Some preparation of a text file with a list of all Oracle table names to be extracted and loaded into PostGIS is required.
rem This reads a list of table names from txt file and passes them to ogr2ogr which then exports from Oracle into PostGIS
for /F %%f in (C:\path\to\your\textfile\OracleTables.txt) do ogr2ogr -a_srs EPSG:27700 -overwrite -f "PostgreSQL" -nln yourschema.%%f PG:"host=localhost user=youruser password=yourpassword dbname=yourdatabase" OCI:"oracleuser/oraclepassword@(DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = oraclehost)(PORT = 1521)))(CONNECT_DATA = (SID =oraclesid))):oracleschema.%%f"
Traditionally exporting from Oracle to PostGIS involved an intermediate step Oracle to Shapefile and then Shapefile to PostGIS. But this means that field names longer than 10 characters in Oracle get truncated in the conversion to Shapefile. And using the QGIS or PgAdminIII shp2pgsql gui has no easy way to set the schema or SRID other than manual intervention. I guess it could be run from the command line though…
Using Safe’s FME you can go directly from Oracle to PostGIS. But there is no way to define the destination schema in the PostGIS database. The PostGIS writer expects there to be a schema with the same name as the Oracle schema in order to complete the extraction and translation. See https://groups.google.com/forum/#!topic/fmetalk/zQsUUJNR5h8
So, open-source here we come. ogr2ogr can read and write to a number of different formats including Oracle (this uses the proprietary OCI.dll). On Windows install OSGeo4W with GDAL/OGR. Run the setup.exe and “Advanced Install” to install the gdal-oracle10g package. See http://stackoverflow.com/questions/9789686/migrate-spatial-data-from-oracle-to-postgresql
The command to issue at the command line is (all on one line):
ogr2ogr -a_srs <srs> -overwrite -f "PostgreSQL" -nln <postgis schema>.<postgis table> PG:"host=<postgis host> user=<postgis user> password=<postgis password> dbname=<postgis database>" OCI:"<oracle user>/<oracle password>@(DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(HOST = <hostname>)(PORT = <port number>)))(CONNECT_DATA = (SID =<sid name>))):<oracle schema>.<oracle table>"
<srs> = in the EPSG format e.g EPSG:27700
This uses ogr2ogr to connect to an Oracle database, pull a table from a schema and write it to a defined schema in PostGIS database.
This is the challenge for this week.
PostgreSQL + PostGIS + MapServer + MapProxy + Quantum GIS on Windows 7 32-bit
The aim is to create a scalable mapping solution based on open-source software that is easy to use and maintain, is fast and responsive and that can provide data in a number of different formats. Windows installers are available for most of the applications I wanted to use which made life a bit easier. Doing the same thing on Linux might be even easier. The corporate proxy and firewall certainly made things more complicated than they should have been.
I am running Fedora 16 with KDE 4.8.3 and Amarok 2.5.0. I have a last.fm account but I couldn’t get Amarok to scrobble my tracks. It turns out it is an issue with KWallet, the KDE password manager.
The solution was found here in the KDE forums and, while this references version 2.3.x and the problem persists in 2.5.0, the solution is the same.
Restart Amarok and start playing. Check your last.fm account and see your currently playing tracks. It works!
For Christmas Kevin gave me a WH1081 wireless USB weather station. It has a wind vane and anemometer, minimum and maximum thermometer, barometer, rain gauge and hygrometer mounted externally. It communicates wirelessly with a base station inside the house. The base station is connected to the server in the loft and there is much chatter between the two. The server runs pywws (Python Wireless Weather Station) software developed by Jim Easterbrook. This processes the data from the weather station at regular intervals and submits the information to a number of different sites. You can check the weather for IPERTHSH12 on Weather Underground (wunderground.com) and on the Met Office WOW site (wow.metoffice.gov.uk). You can also follow the current weather and 12 hourly forecasts on twitter (@newtyleweather). I also wrote up my own pages to display the weather information (cribbing from various other sites on the web) and give further details here: http://wx.mixedbredie.net/ Continue reading
Normally upgrading WordPress is as simple as clicking the Updates button. This is a pleasure when you have six live sites and two test/dev sites. The latest upgrade to version 3.3 from 3.2.1 was not as straightforward as is normal. Updates informed me there were several plugins and themes with upgrades available. OK, quick scan to check what was changing, check all, upgrade. Voila! It is done. Backup datebase. Export blog XML. Ready to move from 3.2.1 to 3.3… Continue reading