Question

MATLAB Image Stitching

Code for Panorama image stitching

either using MATLAB or Python

Answer 1

Answered below both MATHLAB and PHYTON image stiching code

MATHLAB code for feature based image stiching

Step 1: Load Images

% Load images. buildingDir = fullfile(toolboxdir('vision'), 'visiondata', 'building'); buildingScene = imageDatastore(buildingDir); % Display images to be stitched montage(buildingScene.Files)

Step 2: Register Image Pairs

To create the panorama, start by registering successive image pairs using the following procedure:

1. Detect and match features between I(n) and I(n−1).
2. Estimate the geometric transformation, T(n), that maps I(n) to I(n−1).
3. Compute the transformation that maps I(n) into the panorama image as T(n)∗T(n−1)∗...∗T(1).

% Read the first image from the image set. I = readimage(buildingScene, 1); % Initialize features for I(1) grayImage = rgb2gray(I); points = detectSURFFeatures(grayImage); [features, points] = extractFeatures(grayImage, points); % Initialize all the transforms to the identity matrix. Note that the % projective transform is used here because the building images are fairly close to the camera. % Had the scene been captured from a further distance, an affine transform would suffice. numImages = numel(buildingScene.Files); tforms(numImages) = projective2d(eye(3)); % Initialize variable to hold image sizes. imageSize = zeros(numImages,2); % Iterate over remaining image pairs for n = 2:numImages         % Store points and features for I(n-1).     pointsPrevious = points;     featuresPrevious = features;             % Read I(n).     I = readimage(buildingScene, n);         % Convert image to grayscale.     grayImage = rgb2gray(I);            % Save image size.     imageSize(n,:) = size(grayImage);         % Detect and extract SURF features for I(n).     points = detectSURFFeatures(grayImage);        [features, points] = extractFeatures(grayImage, points);     % Find correspondences between I(n) and I(n-1).     indexPairs = matchFeatures(features, featuresPrevious, 'Unique', true);            matchedPoints = points(indexPairs(:,1), :);     matchedPointsPrev = pointsPrevious(indexPairs(:,2), :);                % Estimate the transformation between I(n) and I(n-1).     tforms(n) = estimateGeometricTransform(matchedPoints, matchedPointsPrev,...         'projective', 'Confidence', 99.9, 'MaxNumTrials', 2000);         % Compute T(n) * T(n-1) * ... * T(1)     tforms(n).T = tforms(n).T * tforms(n-1).T; end

Start by using the projective2d outputLimits method to find the output limits for each transform. The output limits are then used to automatically find the image that is roughly in the center of the scene.

% Compute the output limits for each transform for i = 1:numel(tforms)               [xlim(i,:), ylim(i,:)] = outputLimits(tforms(i), [1 imageSize(i,2)], [1 imageSize(i,1)]);    end

Next, compute the average X limits for each transforms and find the image that is in the center. Only the X limits are used here because the scene is known to be horizontal. If another set of images are used, both the X and Y limits may need to be used to find the center image.

avgXLim = mean(xlim, 2); [~, idx] = sort(avgXLim); centerIdx = floor((numel(tforms)+1)/2); centerImageIdx = idx(centerIdx); % apply the center image's inverse transform to all the others Tinv = invert(tforms(centerImageIdx)); for i = 1:numel(tforms)        tforms(i).T = tforms(i).T * Tinv.T; end

Step 3: Initialize the Panorama

Now, create an initial, empty, panorama into which all the images are mapped.

Use the outputLimits method to compute the minimum and maximum output limits over all transformations. These values are used to automatically compute the size of the panorama.

for i = 1:numel(tforms)               [xlim(i,:), ylim(i,:)] = outputLimits(tforms(i), [1 imageSize(i,2)], [1 imageSize(i,1)]); end maxImageSize = max(imageSize); % Find the minimum and maximum output limits xMin = min([1; xlim(:)]); xMax = max([maxImageSize(2); xlim(:)]); yMin = min([1; ylim(:)]); yMax = max([maxImageSize(1); ylim(:)]); % Width and height of panorama. width = round(xMax - xMin); height = round(yMax - yMin); % Initialize the "empty" panorama. panorama = zeros([height width 3], 'like', I);

Step 4: Create the Panorama

Use imwarp to map images into the panorama and use vision.AlphaBlender to overlay the images together.

blender = vision.AlphaBlender('Operation', 'Binary mask', ...  'MaskSource', 'Input port'); % Create a 2-D spatial reference object defining the size of the panorama. xLimits = [xMin xMax]; yLimits = [yMin yMax]; panoramaView = imref2d([height width], xLimits, yLimits); % Create the panorama. for i = 1:numImages         I = readimage(buildingScene, i);          % Transform I into the panorama.     warpedImage = imwarp(I, tforms(i), 'OutputView', panoramaView);                       % Generate a binary mask.        mask = imwarp(true(size(I,1),size(I,2)), tforms(i), 'OutputView', panoramaView);         % Overlay the warpedImage onto the panorama.     panorama = step(blender, panorama, warpedImage, mask); end figure imshow(panorama)

Below sample program stitching.py uses Stitcher API from python to stitch panoramas

# Python 2/3 compatibility from __future__ import print_function import numpy as np import cv2 as cv import argparse import sys modes = (cv.Stitcher_PANORAMA, cv.Stitcher_SCANS) parser = argparse.ArgumentParser(prog='stitching.py', description='Stitching sample.') parser.add_argument('--mode',      type = int, choices = modes, default = cv.Stitcher_PANORAMA,     help = 'Determines configuration of stitcher. The default is `PANORAMA` (%d), '      'mode suitable for creating photo panoramas. Option `SCANS` (%d) is suitable '     'for stitching materials under affine transformation, such as scans.' % modes) parser.add_argument('--output', default = 'result.jpg',      help = 'Resulting image. The default is `result.jpg`.') parser.add_argument('img', nargs='+', help = 'input images') __doc__ += '\n' + parser.format_help() def main():      args = parser.parse_args()      # read input images      imgs = []      for img_name in args.img:          img = cv.imread(cv.samples.findFile(img_name))          if img is None:              print("can't read image " + img_name)              sys.exit(-1)          imgs.append(img)      stitcher = cv.Stitcher.create(args.mode)      status, pano = stitcher.stitch(imgs)      if status != cv.Stitcher_OK:          print("Can't stitch images, error code = %d" % status)          sys.exit(-1)      cv.imwrite(args.output, pano)      print("stitching completed successfully. %s saved!" % args.output)      print('Done') if __name__ == '__main__':      print(__doc__)      main()      cv.destroyAllWindows()