OpenCV: building UWP binaries

Artikel
11/27/2015

[This blog article documents my continuing mission to figure out how to use OpenCV in UWP apps...]

How to build OpenCV native dlls for UWP

OpenCV is on GitHub. There are three relevant forks/branches, as of writing on November 2015:

+- [2] fork https://github.com/Microsoft/opencv

+- [3] branch https://github.com/Microsoft/opencv/tree/vs2015-samples

Repository [1] is the master. In it, the native OpenCV binaries are built for all different platforms using CMake. As far as I can tell, this repository only has support via CMake for building Windows Store and Windows Phone 8.0 and 8.1 apps. It doesn't seem to have UWP. And it also requires you to install some particular old versions of VS, as per its readme.txt.

Fork [2] doesn't seem to bring anything new.

Branch [3] contains Visual Studio .sln files from which you can build UWP versions of all the native OpenCV binaries: x86, x64 and ARM, Debug and Release. The Release builds actually were a bit wonky, but I submitted PR #69 to make them build right. So: if you clone this repository, select this branch, set an environment variable, and open up vs2015/WS/10.0/ARM/OpenCV.sln, or x86/OpenCV.sln, or x64/OpenCV.sln, then you can successfully build UWP binaries for OpenCV. This will produce 15 binaries for each architecture+build combination.

How to use OpenCV native dlls for UWP

I tried out the OpenCV native dlls in a C++ UWP app, and they worked fine.

More interestingly, the dlls all together add up to about 30mb, but I was able to use just three of them that I needed and they worked fine. This satisfies one of my central requirements, to be able to deploy just a cut-down version of OpenCV with my store app, to keep size down.

I did File>New>C++>Windows>UWP blank app. In Solution Explorer I added three DLLs as "content" files in the root directory of my project so they would be copied into the resulting appx, adjacent to the .exe:

opencv_core300d.dll
opencv_igmcodecs300d.dll
opencv_igmproc300d.dll

In Solution Explorer I added three import-libraries. By default, when you add LIBs to a C++ project, they are taken as import libraries:

opencv_core300d.lib
opencv_igmcodecs300d.lib
opencv_igmproc300d.lib

In Project Properties I added an include directory to my C++ project options

vs2015/WS/10.0/include

In my mainpage C++ file, I added a bunch of #include and using statements:

#include <robuffer.h>
#include <opencv2\imgcodecs.hpp>
#include <opencv2\imgproc.hpp>

using namespace Windows::UI::Xaml::Media::Imaging;
using namespace Windows::Storage::Streams;
using namespace Microsoft::WRL;

I added a content file "demo.jpg" in Solution Explorer, and in MainPage.xaml I added an Image control named "image1", and added these lines in OnNavigatedTo in my mainpage C++ file:

cv::Mat bmp1 = cv::imread("demo.jpg");
cv::Mat bmp2 = cv::Mat(bmp1.rows, bmp1.cols, CV_8UC4);
cv::cvtColor(bmp1, bmp2, CV_BGR2BGRA);
WriteableBitmap^ wbmp = ref new WriteableBitmap(bmp2.cols, bmp2.rows);
IBuffer^ buffer = wbmp->PixelBuffer;
unsigned char* dstPixels;
ComPtr<IBufferByteAccess> pBufferByteAccess;
ComPtr<IInspectable> pBuffer((IInspectable*)buffer);
pBuffer.As(&pBufferByteAccess);
pBufferByteAccess->Buffer(&dstPixels);
memcpy(dstPixels, bmp2.data, bmp2.step.buf[1] * bmp2.cols * bmp2.rows);
image1->Source = wbmp;

Result: it all worked. The image displayed correctly.

Next Steps

I want to package each individual DLL up as a NuGet package, for 15 NuGet packages in total. Each package should include x86+x64+ARM versions of the DLLs, and corresponding import libraries. It should also include the include files pertaining to that DLL. That way, as a C++ author, I will be able to bring in only the particular DLLs that I want. To do this I'll have to figure out the dependencies between the DLLs, and the dependencies between header files, and then express these as NuGet dependencies. (I don't yet know how to create NuGet packages for C++. I'll look to "Win2d.uwp" as the canonical example. I don't know if each package should also include debug builds, but I suspect not. I don't know where PDBs should go.)

I think that NuGet generation should be done as part of [3] branch vs2015-samples. This is the branch that has .sln files for building all the DLLs, so it's appropriate for it to also include NuGet-package-building. I don't know if this is within the vision of the owners of that branch or not. Ultimately, in the future, I don't know if the owners of Itseez/opencv would be open to their CMake system building the NuGet packages as well?

Beyond that, the major issue is that OpenCV has a C++ API. This is not friendly to pinvoke interop with .NET. OpenCV used to have a plain C API which is friendly, but the C API has been deprecated and is not getting new features. I don't wish to invent a pinvoke-friendly wrapper around the OpenCV C++ API.

Approach (1): I'd like to test the scenario of a hybrid app, where most of the app is in .NET, and it calls into a C++ winrt component that I author, and this C++ component does all the calls into OpenCV, which it obtains via NuGet.

Approach (2): OpenCVSharp has a BSD license and already contains a set of 'extern "C"' wrappers around the C++ API. The ideal would be for OpenCVSharp to be able to consume my native UWP dlls for OpenCV. There already is a "dll-free" version of OpenCVSharp, which merely contains the pinvoke wrappers, and expects you to supply the OpenCV DLLs some other way. There are a few unknowns with this approach. I don't know if the OpenCVSharp project has different version of OpenCV (and hence different include files) as compared to the one in branch [3]. I don't know if the native extern "C" assembly that comes with OpenCVSharp will be compatible with the DLLs I build, and the fact that there are 15 of them rather than just 1 big "world" dll. I don't know if it will need to switch to "delay-load" to support the scenario where you only package say 3 of the OpenCV dlls with your store app.

Notes

I am hugely excited! It was thrilling to see the UWP versions of all the DLLs get built properly. I think I know how to package up DLLs into NuGet better than most folk, and I can do a best-in-class job. I'm excited that I was able to get something working end-to-end at least in C++.

Comments

Anonymous
February 06, 2016
Very cool. Please keep posting your progress. I would love to be able to use OpenCV in .Net UWP apps
Anonymous
April 20, 2016
I don't know why but I am not able to download your packages from nuget into a Universal Windows phone app that targets version 8.1.
Anonymous
May 09, 2016
Hi, Thabk you very much for tour work, it's awesome.Do you know if exists any progress about opencv on uwp? Or windows phone?I want to add opencv over c# but i can't find concrete info to create a basic project.
Anonymous
May 11, 2016
The comment has been removed
Anonymous
September 22, 2016
Hi Lucian,Could you please help, how to build opencv_contrib module with this +- [3] branch https://github.com/Microsoft/opencv/tree/vs2015-samples branch
- Anonymous
  November 24, 2016
  Hello, i need to use openCv on uwp win 10 iot, ideally in c#, not cpp, is it possible now? Thanks
Anonymous
December 28, 2016
The comment has been removed
Anonymous
June 01, 2017
Hello everybody. I tried to modify this example, add a video capture instead of a photo.cv::VideoCapture cam(0);cv::Mat bmp1;cam >> bmp1;//cv::Mat bmp1 = cv::imread("demo.png");But it produces errors:Exception thrown at 0x0F4E8BB5 (opencv_core300d.dll) in App5.exe: 0xC0000005: Access violation reading location 0xFFFFFFFC.Unhandled exception at 0x0F4E8BB5 (opencv_core300d.dll) in App5.exe: 0xC0000005: Access violation reading location 0xFFFFFFFC.Also tried to change CV_BGR2BGRA on cv :: COLOR_BGR2GRAY, but the program began to work incorrectly. The image did not become gray and instead of 1 image appeared 4Here is a screenshot: https://drive.google.com/open?id=0B3yXw1_qMj1yR0lveXZCNXlCTGsPlease help me solve these problems.
Anonymous
March 11, 2018
Anyone have a sample working for C#? Thank!
Anonymous
February 23, 2019
Very helpful keep posted

Freigeben über