Partager via

OpenCV: first version up on NuGet

  • Article

I got OpenCV to work via NuGet! Here are the steps to consume it:

 

1. File > New > C++ > UWP

 

2. References > Manage NuGet References > Add OpenCV.UWP.native.imgcodecs

This package is currently in pre-release, so it only shows up if you check the "Include Pre-release" checkbox.

It brings in two other dependencies: imgproc and imgcore.

 

3. Write some code! Here are some starters: add these to your #include and your using sections in MainPage.xaml.cpp:

#include <robuffer.h>
#include <opencv2\imgcodecs.hpp>
#include <opencv2\imgproc.hpp>

using namespace Windows::UI::Xaml::Media::Imaging;
using namespace Windows::Storage::Streams;
using namespace Microsoft::WRL;

Next add this inside your MainPage constructor, just after InitializeComponents:

cv::Mat bmp1 = cv::imread("demo.jpg");
cv::Mat bmp2 = cv::Mat(bmp1.rows, bmp1.cols, CV_8UC4);
cv::cvtColor(bmp1, bmp2, CV_BGR2BGRA);
WriteableBitmap^ wbmp = ref new WriteableBitmap(bmp2.cols, bmp2.rows);
IBuffer^ buffer = wbmp->PixelBuffer;
unsigned char* dstPixels;
ComPtr<IBufferByteAccess> pBufferByteAccess;
ComPtr<IInspectable> pBuffer((IInspectable*)buffer);
pBuffer.As(&pBufferByteAccess);
pBufferByteAccess->Buffer(&dstPixels);
memcpy(dstPixels, bmp2.data, bmp2.step.buf[1] * bmp2.cols * bmp2.rows);
image1->Source = wbmp;

Next, add an Image to your MainPage.xaml:

<Image x:Name="image1"/>

Next find a picture, rename it "demo.jpg", and add it to your project. (By default when you drag a jpg into SolutionExplorer it gets placed in the Assets folder; the code above assumes instead that it's at top level).

 

4. Run it!

What's exciting is that this will run on all x86|x64|ARM configurations, and on all Debug|Release. It will pick the correct OpenCV binaries+libraries for the configuration you select.

 

 

5. Debugging: download the symbols

I tried to upload the PDBs to symbolsource.org as recommended in the NuGet docs, but symbolsource.org seems to be down or not accepting symbols. So instead, download them from here, and unzip onto your desktop:

  • OpenCV.UWP.native.symbols.zip [145mb; right-click > save-as].
  • Once it's downloaded, right-click > Properties > Unblock and then right-click > ExtractAll

 

6. Debugging: step into the OpenCV source code

The steps here are kind of fiddly, so that video might be the easiest way to follow them. It shows every single step explicitly.

  1. The first time you debug, go to Debug > Windows > Modules, sort this list alphabetically by the Name column, look for opencv_*, right-click > LoadSymbolInformation, and point it to the symbol directory you unzipped.
  2. Tools > Options > Debugging > uncheck the "Just My Code" button
  3. Tools > Options > Debugging > "Enable source server support".

Once that was set up, I wanted to F11 into my call to "cv::Mat(bmp1.rows, bmp1.cols, CV_8UC4);". At first when I F11 it just stepped through header files, until it got to this line in mat.inl.hpp:

create(2, sz, _type);

This particular function isn't inline in a header file; it resides in the DLL. There's something I don't understand about C++. How come, when I F11 now it just skips over that function? And when I right-click step-into-specific then it says no source is available? but then when I hit F11 a second time then it jumps into the source code proper? Anyway, here's how to make ti work. I did right-click > Step Into Specific > cv::Mat::create. This claims the source isn't available, so I click on view disassembly, to this line:

00398F48 jmp dword ptr [__imp_cv::Mat::create (03F2490h)]

If I hit F11 a second time then it downloads the OpenCV source code file automatically from GitHub (using the "enable source server support" feature that you checked earlier, and a clever technique by Hamish Graham to use GitHub itself as the source server). You can close the disassembly now and double-click on the top frame in the callstack. Now you can step through the source code of the DLL.

 

 

 So far I've only put three of the 15 OpenCV modules on NuGet: core, imgproc and imgcodecs. The work I did to generate these NuPkgs is here on GitHub.

Comments

  • Anonymous
    December 15, 2015
    Can I also access those dll's in C#?

  • Anonymous
    December 15, 2015
    Hi, I was using Win2d using ChromaKeyEffect and I successfully removed the background more or less accurate with the parameters, tolerance and else, so I took a look about what has been done in other platforms and I discovered that in Android they use this OpenCV for the Grabcut  effect. (All discovered yesterday) just 15 days after you release this wonderful version in Nuget. I am really noob on this. I hope the python scripts are reasonable esay to port to C++ library and then use in my C# apps. Do you think is possible to create the GrabCut effect with this release?. Thanks a lot for releasing this!. @juanpaexpedite

  • Anonymous
    December 16, 2015
    I feel very incompetent, I cannot even add the first Scalar classes, is needed any other library? github.com/.../grabcut.cpp, thanks as always this will take a lot of time to unverstand this

  • Anonymous
    December 16, 2015
    Sorry I thought that Intellisense solves cv::Scalar and cv::Mat but not, so forget my previous comment (I do not have any idea to delete a comment).

  • Anonymous
    December 16, 2015
    @MaWo - my chief goal is to use these DLLs from VB/C# in UWP apps. But I'm a way off that -- my next step is to just spend time learning OpenCV in C++, discovering what it's like, making sure I'm on the right track. I expect to get to C# around March 2016. @Juanpaexpedite - I've never heard of Grabcut before but the Wikipedia page makes it sound pretty powerful. Please let us know how you get on!

  • Anonymous
    December 17, 2015
    Hi I have just tweeted at my account a video @juanpaexpedite with my first test of the Grabcut, it is awesome, just one iteration and works like a charm. My mind needs a rest, tomorrow I will publish better video about that.

  • Anonymous
    January 06, 2016
    The comment has been removed

    • Anonymous
      March 16, 2016
      It works, It is possible to use videoCapture ? I dont think so.

  • Anonymous
    August 11, 2016
    Hi Lucian,When generating opencv from source using Cmake it removes opencl, tbb and ipp and also does not make winrt with System_name = WindowsStore and System_version = 10.0. Also highgui functions fir winrt not incorporated. Do you have any ideas or advise to change the Cmake file to allow for opencl and also include the winrt module with highgui winrt additions?. My application requires camera frame processing with some speed for body detection. This will then trigger further processing routines using the Microsoft project oxford vision and face api's

  • Anonymous
    September 04, 2016
    I'm pretty sure I followed the steps in the video exactly, using the code given on this page, and received these errors:https://collindavis0.files.wordpress.com/2016/09/errorcap1.jpgIf you can offer any help, I would appreciate it.

    • Anonymous
      September 04, 2016
      @Collin - oops it looks like my blog post got formatted with "smart quotes". Please try replacing the smart quotes with regular quotes in your strings.
      • Anonymous
        September 04, 2016
        Well well, I was outsmarted by smart quotes. Thanks!
      • Anonymous
        September 04, 2016
        Awesome, man. Just got it to deploy to a Raspberry Pi.

  • Anonymous
    October 09, 2016
    The comment has been removed

  • Anonymous
    October 14, 2016
    Hi, Just want to ask if there is already a C# wrapper for UWP?Thanks!Jude Alquiza

  • Anonymous
    November 07, 2016
    The comment has been removed

  • Anonymous
    November 07, 2016
    The comment has been removed