You can find the full code here:github.com/KorayGocmen/medianfilter
Median filtering can be used to join multiple images to get a single image with all moving objects removed. The simple idea is to compare the pixels at the same location on all images and get the median of RGBA values of the pixels to form a new composite image with all moving objects removed.
The image set provided has to be taken by a camera that did not move during the shoot, since the pixels at the same location on all images have to correspond to the same physical location. Otherwise the median filter will blend all images together to create a meaningless composite image.
You can read more about median filtering on Nikolas Moya's medium article. (I took the test frames I used from this article as well)Simple algorithm to remove moving objects from pictures
The following code is the definition of the image struct I am going to be using, a few utility functions to read an image from a file to my structs and functions to write the struct back to a proper image file. This code is exactly the same code I used in my 3 part article "Writing an image manipulation library in Go". You can find more information about these functions and structs
hereWriting an image manipulation library in Go - Part 1
Utility functions - read above
// Pixel is a single pixel in 2d array
type Pixel struct {
R int
G int
B int
A int
}
// Image is the main object that holds information about the
// image file. Also is a wrapper around the decoded image
// from the standard image library.
type Image struct {
Pixels [][]Pixel
Width int
Height int
_Rect image.Rectangle
_Image image.Image
}
// set pixel value with key name and new value
func (pix *Pixel) set(keyName string, val int) Pixel {
switch keyName {
case "R":
pix.R = val
case "G":
pix.G = val
case "B":
pix.B = val
case "A":
pix.A = val
}
return *pix
}
// rgbaToPixel alpha-premultiplied red, green, blue and alpha values
// to 8 bit red, green, blue and alpha values.
func rgbaToPixel(r uint32, g uint32, b uint32, a uint32) Pixel {
return Pixel{
R: int(r / 257),
G: int(g / 257),
B: int(b / 257),
A: int(a / 257),
}
}
// newImage reads an image from the given file path and return a
// new `Image` struct.
func newImage(filePath string) (*Image, error) {
s := strings.Split(filePath, ".")
imgType := s[len(s)-1]
switch imgType {
case "jpeg", "jpg":
image.RegisterFormat("jpeg", "jpeg", jpeg.Decode, jpeg.DecodeConfig)
case "png":
image.RegisterFormat("png", "png", png.Decode, png.DecodeConfig)
default:
return nil, errors.New("unknown image type")
}
imgReader, err := os.Open(filePath)
if err != nil {
fmt.Println("error opening")
return nil, err
}
img, _, err := image.Decode(imgReader)
if err != nil {
fmt.Println("error decoding")
return nil, err
}
bounds := img.Bounds()
width, height := bounds.Max.X, bounds.Max.Y
var pixels [][]Pixel
for y := 0; y < height; y++ {
var row []Pixel
for x := 0; x < width; x++ {
pixel := rgbaToPixel(img.At(x, y).RGBA())
row = append(row, pixel)
}
pixels = append(pixels, row)
}
return &Image{
Pixels: pixels,
Width: width,
Height: height,
_Rect: img.Bounds(),
_Image: img,
}, nil
}
// writeToFile writes iamges to the given filepath.
// Returns an error if it occurs.
func (img *Image) writeToFile(outputPath string) error {
cimg := image.NewRGBA(img._Rect)
draw.Draw(cimg, img._Rect, img._Image, image.Point{}, draw.Over)
for y := 0; y < img.Height; y++ {
for x := 0; x < img.Width; x++ {
rowIndex, colIndex := y, x
pixel := img.Pixels[rowIndex][colIndex]
cimg.Set(x, y, color.RGBA{
uint8(pixel.R),
uint8(pixel.G),
uint8(pixel.B),
uint8(pixel.A),
})
}
}
s := strings.Split(outputPath, ".")
imgType := s[len(s)-1]
switch imgType {
case "jpeg", "jpg", "png":
fd, err := os.Create(outputPath)
if err != nil {
return err
}
switch imgType {
case "jpeg", "jpg":
jpeg.Encode(fd, cimg, nil)
case "png":
png.Encode(fd, cimg)
}
default:
return errors.New("unknown image type")
}
return nil
}
The following code find the median RGB values of an array of pixels provided and creates a new pixel with the median values.
// medianPixel finds the median r, g, b values from the given
// pixel array and creates a new pixel from that median values
func medianPixel(pixels []Pixel) Pixel {
var (
rValues []int
gValues []int
bValues []int
)
for _, pix := range pixels {
rValues = append(rValues, pix.R)
gValues = append(gValues, pix.G)
bValues = append(bValues, pix.B)
}
sort.Ints(rValues)
sort.Ints(gValues)
sort.Ints(bValues)
rMedian := rValues[int(len(rValues)/2)]
gMedian := gValues[int(len(gValues)/2)]
bMedian := bValues[int(len(bValues)/2)]
return Pixel{rMedian, gMedian, bMedian, 0}
}
The medianFilter function is the main function that creates an image with each pixel generated from the medianPixel function. It starts by reading all images from a provided array of image paths and creating image objects for each of them. These images has to have the same height and width for this function to work. It then iterates through all pixels locations and creates an image with all pixels generated from medianPixel function.
This function actually manipulates all rows in parallel. By using sync.WaitGroup and go keyword, I am able to do things concurrently. I actually wrote a blog post about concurrecy in go.Concurrency and mutex locks in Go
// medianFilter iterates the given filepaths and generates new image
// objects. It then checks to see if all the heights and the widths
// of the images are matching. If they are, each pixel of every image is
// iterated and a median filter is applied to given images. Returns the
// output image object and an error if there is any.
func medianFilter(filePaths []string) (*Image, error) {
var images []*Image
for _, filePath := range filePaths {
img, err := newImage(filePath)
if err != nil {
return nil, err
}
images = append(images, img)
}
if len(images) < 5 {
return nil, errors.New("not enough images to perform noise reduction")
}
outputImage := images[0]
heigth := outputImage.Height
width := outputImage.Width
for _, img := range images {
if heigth != img.Height || width != img.Width {
return nil, errors.New("at least one image has a different width or height")
}
}
var wg sync.WaitGroup
for rowIndex := 0; rowIndex < heigth; rowIndex++ {
wg.Add(1)
go (func(rowIndex int) {
for colIndex := 0; colIndex < width; colIndex++ {
var pixels []Pixel
for _, img := range images {
pixels = append(pixels, img.Pixels[rowIndex][colIndex])
}
medPixel := medianPixel(pixels)
outputImage.Pixels[rowIndex][colIndex].set("R", medPixel.R)
outputImage.Pixels[rowIndex][colIndex].set("G", medPixel.G)
outputImage.Pixels[rowIndex][colIndex].set("B", medPixel.B)
}
wg.Done()
})(rowIndex)
}
wg.Wait()
return outputImage, nil
}
Finally putting everything together under one exported package function that reads an array of image filepaths, performs median filtering and writes the output image into a new file specified in outputPath.
// RemoveMovingObjs iterates the given filepaths and generates new image
// image that does not have the moving objects in the given images.
func RemoveMovingObjs(filepaths []string, outputPath string) error {
img, err := medianFilter(filepaths)
if err != nil {
return err
}
img.writeToFile(outputPath)
return nil
}
Using image filtering we are able to turn this:
20 frames of this gif was used to create the input images
Into this:
Median filtered image
That's pretty much it. I wasn't expecting something this cool to be that easy. This algorithm can be used to build a lot of interesting things. Maybe I would build an app around this some time.
