Unity, Vuforia & ZXing – Barcode scanning in AR games
I’ve been struggling with finding a good barcode scanner/QR code reader that works with Unity and doesn’t kill the framerate. Given that I also have an AR application I thought that I would try to incorporate Vuforia with ZXing to get the best of both worlds and the good news is that it works pretty well and is very fast. I’m attaching a download here of the package that contains a Unity friendly version of the C# port of ZXing together with my mulithreading manager Loom class. Then here is an example of a script that will scan barcodes and/or QR codes and fire an event and send a message to a target object when something is decoded.
using UnityEngine;
using System.Collections;
using System.Collections.Generic;
using System;
using System.Linq;
using com.google.zxing.qrcode;
using com.google.zxing.multi;
using com.google.zxing.common;
using com.google.zxing;
using com.google.zxing.client.result;
[AddComponentMenu("System/VuforiaScanner")]
public class VuforiaScanner : MonoBehaviour
{
public QCARBehaviour vuforia;
public bool barcode;
public GameObject target;
public static event Action<ParsedResult, string> Scanned = delegate {};
public static event Action<string> ScannedQRCode = delegate {};
public static event Action<string> ScannedBarCode = delegate {};
bool decoding;
bool initGray;
MultiFormatReader mfReader = new MultiFormatReader();
QRCodeReader qrReader = new QRCodeReader();
void Update()
{
if(vuforia == null)
return;
if(vuforia.enabled && !initGray)
{
//Wait 1/4 seconds for the device to initialize (otherwise it seems to crash sometimes)
initGray = true;
Loom.QueueOnMainThread(()=>{
initGray = CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.GRAYSCALE, true);
}, 0.25f);
}
if (vuforia.enabled && CameraDevice.Instance != null && !decoding)
{
var image = CameraDevice.Instance.GetCameraImage(Image.PIXEL_FORMAT.GRAYSCALE);
if (image != null)
{
decoding = true;
Loom.RunAsync(() =>
{
try
{
var bitmap = new BinaryBitmap(new HybridBinarizer(new RGBLuminanceSource(image.Pixels, image.BufferWidth, image.BufferHeight, true)));
Result data;
if (barcode)
{
var reader = new MultiFormatReader();
data = reader.decode(bitmap);
}
else
{
var reader = new QRCodeReader();
data = reader.decode(bitmap);
}
if (data != null)
{
Loom.QueueOnMainThread(() => {
if (data.BarcodeFormat == BarcodeFormat.QR_CODE)
{
ScannedQRCode(data.Text);
if(target != null)
{
target.SendMessage("ScannedQRCode", data.Text, SendMessageOptions.DontRequireReceiver);
}
}
if (data.BarcodeFormat != BarcodeFormat.QR_CODE)
{
ScannedBarCode(data.Text);
if(target != null)
{
target.SendMessage("ScannedBarCode", data.Text, SendMessageOptions.DontRequireReceiver);
}
}
var parsedResult = ResultParser.parseResult(data);
if(target != null)
{
target.SendMessage("Scanned", parsedResult, SendMessageOptions.DontRequireReceiver);
}
Scanned(parsedResult, data.Text);
decoding = false;
});
}
}
catch (Exception e)
{
decoding = false;
}
});
}
}
}
}
Take a look at the implementation of the class InvertedLuminanceSource from ZXing.Net. Since the current version 0.11 it supports decoding of inverted barcodes like your QR code.
https://zxingnet.svn.codeplex.com/svn/trunk/Source/lib/InvertedLuminanceSource.cs
me again
I now realized that it only scans qr with black dots and white bg… and I fail miserably to edit the code to apply a negative filter to the image because my qr graphism are more like the opposite. I am already very close, I am trying to modify the RGBLuminanceSource() luminance data but with no luck…I am not too sure what to apply mathematically to make it work.
Would you happen to know how to do that so that this kind of qr can be scanned?
https://dl.dropboxusercontent.com/u/17356625/Unity/projects/misc/test.png
The annoying thing is that the same qr but negative ( using photoshop), will scan just fine…
Thanks
Jean
hey dude i wanted to ask about if this method will still work with Unity3d fourth new edition ??? thanks in advace
Hi, It’s really odd. I am on u3.5 and I have everything working almost, it detected the qr code fine but the code fails here in your behavior sample: if (data.BarcodeFormat == BarcodeFormat.QR_CODE) with an unity error: CompareBaseObjects can only be called from the main thread.
Constructors and field initializers will be executed from the loading thread when loading a scene.
Don’t use this function in the constructor or field initializers, instead move initialization code to the Awake or Start function.
I am not good with threads… so maybe you have an idea of what could go wrong here? thanks a lot!
Jean
Sorry just saw this (the site is supposed to be putting comments in my forum, so I’ve no idea why its ended up here – must have been a WordPress upgrade issue).
If you haven’t managed to fix this – it would be down to using Loom.QueueOnMainThread before comparing the objects or comparing something to null.
Hi, we’re playing cat and mouse
indeed I was able to solve the problem, by simply not relying on the Loom.QueueOnMainThread((). It simply didn’t work for me for some very odd reasons. so what I do is simply save the result in VuforiaScanner class and then check on a regular MonoBehaviour.update()
thanks
what does it means by installed version of “Vuforia”…..i just import your package and then copy above class into “VuforiaScanner” and place it on GameObject…..Here is the error
Assets/VuforiaScanner.cs(15,12): error CS0246: The type or namespace name `QCARBehaviour’ could not be found. Are you missing a using directive or an assembly reference?
Can you please reply me all the steps for that
You need to install Vuforia from Qualcomm it’s on their website.
Hello ,, first thanks u ALLLOOT u made the app alot easier to me ! ,,,, i used to port from Unity to Eclipse for the QR part ,,, but now should i create new scene for the QR scanner ,,, if so how to integrate the script to the scene ,,,and how to make like a framewith the size if the qr overlayed to the camera ??
thanks
Ahmed Tawheed
I just have a button to start the Vuforia scanner and I don’t have anything in the scene tracked so it just appears as a picture (with my NGUI toolbar at the top)
i want to make a reciever for scannedQRcode to display data as popup message like Toast in Android,,,, in the target object i made the script and added the function scannedQRcode as to listen to a reciever but i can’t call it as object is implicitly called in Unity c# i can’t call it in start() method i think
Hiya,
When I try and use the script above with the project, I get the following error
error CS0246: The type or namespace name `QCARBehaviour’ could not be found. Are you missing a using directive or an assembly reference?
Any ideas please?
And thanks
You need to have installed a version of Vuforia that contains the QCARBehaviour script.
Yeah figured it out, doh! Just came back to post and say how stupid I’d been. But thanks anyway!
Thank you for your brilliant work. I managed to show the QR text easily on the GUI Text. Can you give me a hint that how I can show an AR Object on top of the QR code? Also is it possible for you to update your Unity friendly package with ZXing.net or advice me on how to do that as well?
Thanks a lot again.
Warmest Regards,
Shahram
Unfortunately the AR tracking and the QR codes are two separate systems. I use a QR code to identify the application and then use standard Vurforia tracker markers to locate the scene in the 3D world. So the player scans a QR code and then points at a tracker marker. The QR code can be embedded in the marker.
I’m not sure how you would incorporate ZXing.net yet – I was going to look at that but have become busy on other things. If you just use it as it is, it should provide all of the functionality of bar code reading and QR scanning – which is why I haven’t felt the need to change it yet.
What kind of frame rate are you seeing with this?
When I’m scanning for QR codes it is a little hard to tell given that much of my geometry is disabled so I’m getting 30fps.
Thank you for your brilliant work!!!
How would you then send the read string to a GUIText?
I’m trying to make it work on an iphone…
Ciao!
Giovanni
Thanks
So if you use that script in the article the easiest way to do what you want is to set the “target” variable in the inspector to the object that contains your GUIText and then write a script with a ScannedQRCode function that you attach to that object.
Something like this:
public GUIText qrCodeLabel;
void ScannedQRCode(string text)
{
qrCodeLabel.text = text;
}
Or ScannedBarCode or both depending on what you want!
Hi Mike,
nice solution.
I took your code and modified it so that it works with ZXing.Net.
ZXing.Net is an up-to-date port of zxing. ZXing.Net was created and is maintained by me.
Since version 0.7 ZXing.Net provides a Unity-friendly assembly.
I couldn’t test the functionality of my version of your sample yet but it compiles successfully.
You can find it here if you are interested:
https://zxingnet.svn.codeplex.com/svn/trunk/Clients/VuforiaDemo
If not or if it is not ok for you drop me an email and feel free to remove my comment.
Regards,
MIchael
Brilliant – wish I’d found that before