IMG_EFFECT_OVERLAY doesn't work in PHP 5.6.0 with GD version: 2.1.0-alpha.

^^ my name is coreyward ^^

imagelayereffect needs to be applied to the destination image, not the source image. As noted, the only useful "effect" as of this time is IMG_EFFECT_OVERLAY since the others are merely another way of setting the alpha blending flag. 

You can also use imagecopymerge with a lower pct argument to blend two images with more control over the opacity at which the top image is blended. 

<?

// overlays $im_src onto $im_dst properly (assuming same dims)

imagelayereffect($im_dst, IMG_EFFECT_OVERLAY);
imagecopy($im_dst, $im_src, 0, 0, 0, 0, $im_dst_x, $im_dst_y);

?>

I found the following example on a Chinese site and translated and adjusted it. It creates a portrait from the sample picture.

<?php

// Portrait image
$im = imagecreatefromjpeg( "sample.jpg");

// Get width and height
$w = imagesx($im); 
$h = imagesy($im);

// Non-through
$c_visible = 0x007F7F7F;
// Background (black) 
$c_bg = 0x00000000; 

// Create mask
$im_mask = imagecreatetruecolor($w,$h);
imageFilledRectangle($im_mask, 0,0, $w,$h, $c_bg); 
imageFilledEllipse ($im_mask, $w/2,$h/2, $w*.9,$h*.9, $c_visible);

// Only overlay seems to something useful
imagelayereffect($im_mask, IMG_EFFECT_OVERLAY);

imagecopy($im_mask, $im, 0,0, 0,0, $w,$h); 
imagedestroy($im);

imagepng($im_mask);
imagedestroy($im_mask);

?>

Try removing the 'create mask' code and just add another image with 'createimagefromjpeg' to see 'normal' overlay. It blends the colors like different layers in Photoshop.

I have been getting a fatal error:

Fatal error: Call to undefined function imagelayereffect()

This doesn't work with Ubuntu server (8.04 lts) and the php5-gd package.  If you need this you will have to recompile PHP with GD built-in.