Creating Panoramas and Image Stitching with Pillow

Image stitching is a technique that combines multiple images to create a single panoramic view. The process involves aligning and blending overlapping images to ensure seamless transitions. Understanding the basics begins with recognizing the importance of feature detection and matching.

Key algorithms for feature detection include SIFT, SURF, and ORB. These algorithms identify distinctive points in the images that can be used to align them. Here’s a simple implementation using OpenCV to detect keypoints and descriptors:

import cv2

# Load images
img1 = cv2.imread('image1.jpg')
img2 = cv2.imread('image2.jpg')

# Convert to grayscale
gray1 = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
gray2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)

# Initiate ORB detector
orb = cv2.ORB_create()

# Find keypoints and descriptors
keypoints1, descriptors1 = orb.detectAndCompute(gray1, None)
keypoints2, descriptors2 = orb.detectAndCompute(gray2, None)

Once you have your keypoints and descriptors, the next step is to match them. That’s typically done using a matcher algorithm, such as the BFMatcher or FLANN. Here’s how you can implement matching with BFMatcher:

# Create BFMatcher object
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)

# Match descriptors
matches = bf.match(descriptors1, descriptors2)

# Sort them in ascending order of distance
matches = sorted(matches, key=lambda x: x.distance)

# Draw the matches
img_matches = cv2.drawMatches(img1, keypoints1, img2, keypoints2, matches[:10], None, flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)
cv2.imshow('Matches', img_matches)
cv2.waitKey(0)
cv2.destroyAllWindows()

After matching, the next stage is to estimate the homography matrix, which transforms the coordinates of one image to align with another. That’s critical for stitching the images together effectively. Using RANSAC can help filter out outliers that may disrupt alignment.

# Extract location of good matches
points1 = np.zeros((len(matches), 2), dtype=np.float32)
points2 = np.zeros((len(matches), 2), dtype=np.float32)

for i, match in enumerate(matches):
    points1[i, :] = keypoints1[match.queryIdx].pt
    points2[i, :] = keypoints2[match.trainIdx].pt

# Find homography
H, mask = cv2.findHomography(points1, points2, cv2.RANSAC)

With the homography matrix, you can warp one of the images to align it with the other. This involves transforming the pixel coordinates based on the calculated matrix, which can be done using the warpPerspective function in OpenCV.

# Get the dimensions of the images
h1, w1 = img1.shape[:2]
h2, w2 = img2.shape[:2]

# Create the canvas for the stitched image
result = cv2.warpPerspective(img1, H, (w1 + w2, h1))

# Place the second image on the canvas
result[0:h2, 0:w2] = img2
cv2.imshow('Stitched Image', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Understanding these fundamentals will help you grasp the core concepts behind image stitching. Once you have mastered these techniques, you can explore more advanced methods for optimizing and refining your stitched images for better quality and accuracy. Each step enhances the final output, making it more visually appealing and coherent.

Using Pillow for effective panorama creation

To begin optimizing and refining your stitched images, it is essential to address the blending of the overlapping regions. Simple overlaying can lead to visible seams, so employing feathering or multi-band blending techniques can significantly improve the visual quality. Pillow, a popular Python Imaging Library, can help with this process.

First, ensure you have Pillow installed. If not, you can install it using pip:

pip install Pillow

Once you have Pillow, you can use it to blend the overlapping regions of your stitched images seamlessly. Here’s a basic example of how to blend two images using Pillow:

from PIL import Image

# Load the stitched image
stitched_image = Image.open('stitched_image.jpg')
overlay_image = Image.open('image2.jpg')

# Create a mask for blending
mask = Image.new('L', stitched_image.size, 0)
mask.paste(255, (0, 0, overlay_image.width, overlay_image.height))

# Blend the images
blended_image = Image.composite(stitched_image, overlay_image, mask)
blended_image.show()

In this example, the Image.composite function is used to blend the stitched image with the overlay image based on the mask. The mask determines the transparency of the overlay, allowing for a smooth transition. You can adjust the mask to control how much of each image is visible.

Another method to enhance the quality of your panorama is to correct for color variations between the images. This can be achieved by adjusting brightness, contrast, and color balance. Pillow also provides functions for these adjustments:

from PIL import ImageEnhance

# Enhance brightness
enhancer = ImageEnhance.Brightness(stitched_image)
enhanced_image = enhancer.enhance(1.2)  # Increase brightness by 20%

# Enhance contrast
enhancer = ImageEnhance.Contrast(enhanced_image)
enhanced_image = enhancer.enhance(1.3)  # Increase contrast by 30%

# Show the enhanced image
enhanced_image.show()

Implementing these enhancements can dramatically improve the visual allure of your stitched panorama. Additionally, consider using techniques such as histogram equalization to enhance the overall tonal range of the images.

import numpy as np

# Convert images to numpy arrays for histogram equalization
img_array = np.array(enhanced_image)

# Perform histogram equalization
img_equalized = cv2.equalizeHist(cv2.cvtColor(img_array, cv2.COLOR_RGB2GRAY))

# Convert back to PIL for displaying
equalized_image = Image.fromarray(img_equalized)
equalized_image.show()

Finally, resizing the final stitched image can also play an important role in its presentation. Depending on where the panorama will be displayed, you may want to scale it appropriately. The resize function in Pillow allows you to specify the new dimensions:

# Resize the final image
final_image = enhanced_image.resize((width, height), Image.ANTIALIAS)
final_image.show()

By incorporating these optimization techniques, you can significantly enhance the visual quality of your stitched images, making them more suitable for various applications, from personal projects to professional presentations. Each adjustment contributes to a more polished final product that can stand up to scrutiny in a wide range of contexts.

Optimizing and refining your stitched images

To further refine your stitched images, consider implementing advanced blending techniques. One effective method is using multi-band blending, which allows for a more seamless transition between overlapping images by blending them at different frequency bands. This technique effectively minimizes visible seams and ensures that the colors and textures appear more uniform.

Multi-band blending involves decomposing the images into various frequency bands, blending these bands separately, and then reconstructing the final image. You can achieve this using the OpenCV library in conjunction with NumPy. Here’s a simplified version of how to implement multi-band blending:

import cv2
import numpy as np

def blend_images(img1, img2):
    # Create Gaussian pyramids for both images
    g1 = img1.copy()
    g2 = img2.copy()
    gp1 = [g1]
    gp2 = [g2]
    
    for i in range(6):
        g1 = cv2.pyrDown(g1)
        g2 = cv2.pyrDown(g2)
        gp1.append(g1)
        gp2.append(g2)

    # Create Laplacian pyramids
    lp1 = [gp1[5]]  # Last level is the same
    lp2 = [gp2[5]]
    
    for i in range(5, 0, -1):
        # Subtract the expanded lower level from the upper level
        l1 = cv2.subtract(gp1[i-1], cv2.pyrUp(gp1[i]))
        l2 = cv2.subtract(gp2[i-1], cv2.pyrUp(gp2[i]))
        lp1.append(l1)
        lp2.append(l2)

    # Blend the Laplacian pyramids
    blended_pyramid = []
    for l1, l2 in zip(lp1, lp2):
        blended = cv2.addWeighted(l1, 0.5, l2, 0.5, 0)
        blended_pyramid.append(blended)

    # Reconstruct the blended image
    blended_image = blended_pyramid[0]
    for i in range(1, len(blended_pyramid)):
        blended_image = cv2.pyrUp(blended_image)
        blended_image = cv2.add(blended_image, blended_pyramid[i])

    return blended_image

# Example usage
blended_result = blend_images(img1, img2)
cv2.imshow('Blended Image', blended_result)
cv2.waitKey(0)
cv2.destroyAllWindows()

In addition to blending, consider sharpening your final image. Sharpening enhances edges and details, making the panorama more visually striking. The filter method in Pillow allows you to apply various filters, including sharpening:

from PIL import ImageFilter

# Apply sharpening filter
sharpened_image = final_image.filter(ImageFilter.SHARPEN)
sharpened_image.show()

Another optimization to consider is the removal of lens distortion, which can occur due to the camera’s optics. This distortion can lead to noticeable aberrations in the stitched image. OpenCV provides functions to correct for lens distortion based on camera calibration parameters. If you have these parameters, you can apply the following correction:

# Assume camera_matrix and distortion_coefficients are known
h, w = img1.shape[:2]
new_camera_matrix, roi = cv2.getOptimalNewCameraMatrix(camera_matrix, distortion_coefficients, (w, h), 1, (w, h))

# Undistort images
undistorted_img1 = cv2.undistort(img1, camera_matrix, distortion_coefficients, None, new_camera_matrix)
undistorted_img2 = cv2.undistort(img2, camera_matrix, distortion_coefficients, None, new_camera_matrix)

Applying these optimizations will not only enhance the aesthetic quality of your stitched images but will also ensure that they’re more accurate representations of the scene captured. Each refinement step contributes to a more polished output, ready for any application, from artistic displays to technical presentations.

Source: https://www.pythonlore.com/creating-panoramas-and-image-stitching-with-pillow/

You might also like this video