PHP Tricks: Eliminate any unwanted characters from a string

Hi all.

I’ve been looking for a handy/graceful way to do this for a while and I thought I’d share this little php trick with you in the hopes that it’ll save you some time if you ever need it. If anyone reading this can quickly convert this example into other languages such as perl, coldfusion, ruby, c, c++ please be my guest and post it as a comment or as a trackback to this blog.

What I wanted to do

I wanted to scrub any characters out of a string that were not alphanumeric. That is, not “a” to “z” and “0″ to “9″. Thankfully, PHP has extensive regular expressions support so that’s what we’ll use.

Why

I needed to store files uploaded through PHP on my Linux machine using a filename chosen by the user. A web site I’m working on requires users to upload media (images, video, music, et. al) and in order to save the files on the hard disk. This functionality prevents code failure by making the filename valid in UNIX filesystems.

The Code

/**
 * Converts a string to a valid UNIX filename.
 * @param $string The filename to be converted
 * @return $string The filename converted
 */
function convert_to_filename ($string) {

  // Replace spaces with underscores and makes the string lowercase
  $string = str_replace (" ", "_", $string);
  $string = str_replace ("..", ".", $string);
  $string = strtolower ($string);

  // Match any character that is not in our whitelist
  preg_match_all ("/[^0-9^a-z^_^.]/", $string, $matches);

  // Loop through the matches with foreach
  foreach ($matches[0] as $value) {
    $string = str_replace($value, "", $string);
  }
  return $string;
}

Usage

Simply call the convert_to_filename function and pass the filename/string along. For example:

$valid_filename = convert_to_filename ($original_filename);

How it works

Almost the entire function is self-explanatory as long as you have a bit of experience with PHP. The part that does the actual deed may need some explanation, however. Especially if you’re relatively new to regular expressions. The part that strips the bad characters from the filename string is started first by this line, which uses a regular expression to make an array called $matches of all of the characters that are NOT a-z, 0-9, a period, or an underscore:

preg_match_all ("/[^0-9^a-z^_^.]/", $string, $matches);

Then, we simply walk through the array replacing the offending character with nothing using the PHP function str_replace.

Functions Used

The functions/constructs used in this article are

PHP Smarty Tip: Speed Up Your Site With ‘trimwhitespace’

If you use the Smarty template engine for your site then you can take advantage of the included output filter Smarty plugin called trimwhitespace. Trimwhitespace is an output filter, which is basically a plugin that runs after your source code has been created but before it gets displayed or put into a cache file.

The premise for trimwhitespace is simple. Most HTML source code for most websites includes whitespace put there by the web developer for readability. Since this whitespace is not required for the website to display and takes up needed bandwidth, we can eliminate it. Enter trimwhitespace.

To enable trimwhitespace, place this command before your Smarty display() function:

$smarty->load_filter('output','trimwhitespace');

An added benefit is that while the user recieves a trimmed copy of your HTML source code, you can still see your source in its fully tabbed and spaced glory by examining your template files. It’s a win-win situation!

HTML Code Before ‘trimwhitespace’:

<table>
  <tr>
    <td>
      Content
    </td>
  </tr>
</table>

HTML Code After ‘trimwhitespace’:

<table>
<tr>
<td>
Content
</td>
</tr>
</table>

See the difference?

PHP: Securing Your Input Forms From MySQL Injection Attacks

Every website has ‘em. Forms. Places for users to enter data into your website. Whether it be a search box, a “Contact Us” form, or variables in the website address, at some point in the flow of your script these suckers are going to touch your database

Oh, that’s no problem — We’ll just take what they type in and run a query in MySQL on it!

WHOA, there! Are you sure you want to do that? Any input from a user should be treated like a nuclear fuel rod. You can handle it, but you’ve got to make sure you do it right. You wouldn’t just pick it up with your bare hands, would you?

Why? Just what are MySQL Injection attacks anyway?

Lets say your database has a table inside called ‘tbl_Users’. Inside ‘tbl_Users’ are a list of your users, which all have usernames, passwords, first names, last names, addresses, etc.

Let’s pretend you’ve got a website and in that website you have an area where users have to log in to gain access to a restricted area. After the user types in their username and password your site will check the database to verify the username and password are correct. If it is correct, your site will provide them access.

The query below is an extremely simplified version of what may be running on your site, though I have seen examples of this before.

SELECT * FROM `tbl_Users` WHERE `username`='".$_POST['username']."' AND `password`='".$_POST['password']."'"

Ugh. I feel dirty just writing that.

There are numerous problems in this example that relate not only to MySQL but to general security hazards. Better solutions would be to verify the username and password separately to make logging login attempts easy and also adding a “salt” to the password.

Though this article deals directly with MySQL injection hazards, it is advisable that you do your research when it comes to login forms and security. There are other hazards out there!

The direct MySQL injection threat is that unscrupulous users (read: bad ones) could enter this into your form:

username: no_one
password: ' OR ''=''

Which would make your query look something like this:

SELECT * FROM `tbl_Users` WHERE `username`='no_one' AND `password`='' OR ''=''

This query would allow that user access to restricted page by logging them in. There are a multitude of other ways this can be dangerous, but this is by far the easiest example.

You may be safe from query stacking, though. MySQL will not allow two queries to be executed in a single function call. To make it simple, consider this example, which will cause MySQL to throw an error:

username: no_one
password: ' OR ''=''; DELETE * FROM `tbl_Users`;

Okay so I’ve got this friend… and his website isn’t secure. What can I do to help him out?

The good news is that with a few precautions, your “friend’s” website will be pretty secure against these types of attacks. I say pretty secure because there is no way to prevent every attack. We can only do our best to increase security to a point to take every realistic precaution to prevent these attacks.

 

#1: Escape your variables!

Using the php function ‘mysql_real_escape_string’ you can “escape” the single quote character from user input. This is probably the easiest method to prevent MySQL injection attacks. It works by adding a backslash (“\”) before each quote that the user enters into their input. So, to use our example from before:

username: hey'there

becomes

username: hey\'there

This effectively stops MySQL injection in its tracks since it not only escapes the single quote (“‘”) character but also all other characters that the baddies can use to hijack your queries.

If you’ve got an array of data coming in, you can use this neat function that I found on the PHP mysql_real_escape_string page (code by “brian dot folts at gmail dot com”). It escapes all of the values in your array with ease.

To escape an array, use this function:

function mysql_real_escape_array($t){
return array_map("mysql_real_escape_string",$t);
}

Then you can call that function easily by passing your array to it:

$your_array = mysql_real_escape_array($your_array);

 

#2: Check the variable type of your input.

This is done by using the php functions “is_numeric()“, “is_string()“, “is_float()“, and “is_int()” to determine if the input the user is sending in is the same type that you were asking for. It’s not perfect, but if you were asking for a number and they sent in a word you know to discard it straight away and return an error thereby entirely avoiding any chance of a MySQL injection attack.

 

#3: Use Prepared Statements.

Consider switching from using mysql_xxx commands in php to MySQL Improved (mysqli). A great document by Zak Greant and Georg Richter cover a lot of the basics and reasoning behind this switch.

All of these things put together will help make your site better equipped to handle malicious injection attacks. I hope this gives you a better indication of what you can do to help secure your websites. I want to personally thank Sven Arild Helleland and exsecror.pip.verisignlabs.com who smacked me upside the head a few times while reviewing this article and helped to steer me in the right direction. Thanks a ton!

Leave a comment or two if this helped you at all or if you have different suggestions on how to secure your code from MySQL injection attacks!

Using FFMpeg to Convert mpg or mov to flv

Hi all.

If you need to use a command-line program to convert between different movie or media formats, it’s likely you’ll end up using ffmpeg. ffmpeg is the de facto standard for converting file formats in *nix and FreeBSD environments.

Using the program from the command-line is like using any other program. There are a large amount of switches and options you can use to change the functionality of ffmpeg, including video resolution, video quality, audio compression and quality, framerate, and more!

There are many ways to download ffmpeg. Many vary on your distribution of Linux or other Operating System. In many cases, ffmpeg may already be installed and ready for you to use. For Mandriva users, ffmpeg is available using urpmi or the “Add/Remove Software” program in the control panel.

In case you simply need to convert a video from mov or mpeg format to Flash Video (flv) here is the command that I use:

# ffmpeg -i <filename.mpg> -deinterlace -ar 44100 -r 25 -qmin 3 -qmax 6 <filename.flv>

This will convert your movie to Flash video (.flv extension) at the same resolution it went in as (for example 480×392 pixels in width and height), deinterlace the video, set the audio frequency to 44100 (high quality), the video framerate to be 25 frames per second, and set the video quality between 3 and 6, which will give you very good yet quickly servable results. If you’re shipping this video on CD or DVD you can set the qmin and qmax values lower (lower = higher quality) but for streaming video this is very, very good and very close to the original without being massive in size.

In fact, the one thing that requires some explanation are the qmin and qmax values. It’s a slightly complicated subject but can be easily explained by thinking of qmin and qmax as how much quality you want to take away from your video, between those two numbers. The minimum qmin and qmax is 1 and the maximum is 31.

If the video size is too large after using this command example and you would like a smaller file, try increasing qmax first until you reach it’s maximum. It’s likely that you will find a happy number in there without having to adjust the qmin value.

If you need to convert several videos, say from a directory, you can use this script available here.

Below is the code in the script file, for reference.

<?php
if ($handle = opendir('.')) {
    while (false !== ($file = readdir($handle))) {
        if ($file != "." && $file != ".." && strpos($file, "mpg") !== false) {
                $new_filename = str_replace(" ", "_", strtolower(str_replace("mpg", "flv", $file)));
                $file = str_replace(" ", "\ ", $file);
                exec("ffmpeg -i $file -deinterlace -ar 44100 -r 25 -qmin 3 -qmax 6 $new_filename");
        }
    }
    closedir($handle);
}
?>

It’s fairly self-explanatory, but the above script simply fetches a list of all of the files in the current directory and executes the ffmpeg program for each file in that list that has the extension “.mpg”.

It should be noted that converting between different video file formats may require a license to do so by the patent holders of that file format. Be sure that you have dotted your “i’s” and crossed your “t’s”.

Good luck!

UPDATE: If you’ve found this article or script handy, please digg it. Thanks!

PHP Tricks: How To Handle Multiple Domains

This is really handy for those of us who have the same code handling multiple sites or multiple sub-domains.

A case in point: When I coded NetBoardz (my free forum hosting service now defunct), I had one codebase handling all 250 forums. How? Simple. When the code runs, it determines which site the user is loading and does different things (like using different databases) dynamically.

How to determine the domain the user is using to view your site:

$domain = $_SERVER['HTTP_HOST'];
if ($domain == "xyz") {
...
} else if ($domain == "uvw") {
...
}

In the example above you can see that we have put the domain that the user has used to view your site into the $domain variable, loading the value from the PHP global variable, $_SERVER. The $_SERVER variable is global, which means you can access it anytime and anywhere in your code.

For more information on PHP’s predefined global variables to see what’s available to you, click here.

How to determine the sub-domain the user is using to view your site:

Sample code is from NetBoardz, which is based off of phpBB 2:

$subdomain = strpos($_SERVER['HTTP_HOST'], ".");
$subdomain = substr($_SERVER['HTTP_HOST'], 0, $subdomain);
$dbname = "nb_".$subdomain;
mysql_select_db($dbname, $sql_link);

Here you can see that we retreived the whole hostname, including the top-level domain and subdomain, then used the PHP functions strpos and substr to take anything before the first dot. For example, the whole hostname “testforum.netboardz.com” passed through this code would end up as “testforum”.

After, we use that subdomain name to calculate which forum database to load. Of course, once you have the domain or subdomain in a variable, you are able to handle your code as you wish!

I hope this small tid-bit of code helps you out in some way. I know that there are tons of these snippets and tips littered across the internet and while I was learning PHP they were really handy. I’ll post more over time, always in this same category.