Facebook Twitter Gplus LinkedIn RSS
formats

Unity, Vuforia & ZXing – Barcode scanning in AR games

I’ve been struggling with finding a good barcode scanner/QR code reader that works with Unity and doesn’t kill the framerate.  Given that I also have an AR application I thought that I would try to incorporate Vuforia with ZXing to get the best of both worlds and the good news is that it works pretty well and is very fast.  I’m attaching a download here of the package that contains a Unity friendly version of the C# port of ZXing together with my mulithreading manager Loom class.  Then here is an example of a script that will scan barcodes and/or QR codes and fire an event and send a message to a target object when something is decoded.

using UnityEngine;
using System.Collections;
using System.Collections.Generic;
using System;
using System.Linq;
using com.google.zxing.qrcode;
using com.google.zxing.multi;
using com.google.zxing.common;
using com.google.zxing;
using com.google.zxing.client.result;

[AddComponentMenu("System/VuforiaScanner")]
public class VuforiaScanner : MonoBehaviour
{
	public QCARBehaviour vuforia;
	public bool barcode;
	public GameObject target;

	public static event Action<ParsedResult, string> Scanned = delegate {};
	public static event Action<string> ScannedQRCode = delegate {};
	public static event Action<string> ScannedBarCode = delegate {};

	bool decoding;
	bool initGray;
	MultiFormatReader mfReader = new MultiFormatReader();
	QRCodeReader qrReader = new QRCodeReader();

	void Update()
	{
		if(vuforia == null)
			return;
		if(vuforia.enabled && !initGray)
		{
                     //Wait 1/4 seconds for the device to initialize (otherwise it seems to crash sometimes)
		     initGray = true;
		     Loom.QueueOnMainThread(()=>{
	                      initGray = CameraDevice.Instance.SetFrameFormat(Image.PIXEL_FORMAT.GRAYSCALE, true);
		     }, 0.25f);
		}
		if (vuforia.enabled && CameraDevice.Instance != null && !decoding)
		{
			var image = CameraDevice.Instance.GetCameraImage(Image.PIXEL_FORMAT.GRAYSCALE);
			if (image != null)
			{
				decoding = true;

				Loom.RunAsync(() =>
				{
					try
					{
						var bitmap = new BinaryBitmap(new HybridBinarizer(new RGBLuminanceSource(image.Pixels, image.BufferWidth, image.BufferHeight, true)));
						Result data;
						if (barcode)
						{
							var reader = new MultiFormatReader();
							data = reader.decode(bitmap);
						}
						else
						{
							var reader = new QRCodeReader();
							data = reader.decode(bitmap);
						}
						if (data != null)
						{
							Loom.QueueOnMainThread(() => {
								if (data.BarcodeFormat == BarcodeFormat.QR_CODE)
								{
									ScannedQRCode(data.Text);
									if(target != null)
									{
										target.SendMessage("ScannedQRCode", data.Text, SendMessageOptions.DontRequireReceiver);
									}
								}
								if (data.BarcodeFormat != BarcodeFormat.QR_CODE)
								{
									ScannedBarCode(data.Text);
									if(target != null)
									{
										target.SendMessage("ScannedBarCode", data.Text, SendMessageOptions.DontRequireReceiver);
									}

								}
								var parsedResult = ResultParser.parseResult(data);
								if(target != null)
								{
									target.SendMessage("Scanned", parsedResult, SendMessageOptions.DontRequireReceiver);
								}
								Scanned(parsedResult, data.Text);
								decoding = false;
							});
						}
					}
					catch (Exception e)
					{
						decoding = false;
					}
				});

			}

		}
	}

}

26 Responses

  1. Murray

    Thanks for this, i’ve been using it for a while but with last Vuforia update (v2.8) it’s not working anymore.

    I suppose thats because of a change in vuforia library, but I’m not sure how to solve that.

    From release notes: (https://developer.vuforia.com/resources/sdk/unity)
    The names of the plugin libraries have changed: libQCAR.so/libQCAR.jar have been renamed respectively to libVuforia.so/libVuforia.jar.

  2. armando

    it doesnt scan the barcode.. the qrcode yes

  3. Take a look at the implementation of the class InvertedLuminanceSource from ZXing.Net. Since the current version 0.11 it supports decoding of inverted barcodes like your QR code.
    https://zxingnet.svn.codeplex.com/svn/trunk/Source/lib/InvertedLuminanceSource.cs

  4. me again :) I now realized that it only scans qr with black dots and white bg… and I fail miserably to edit the code to apply a negative filter to the image because my qr graphism are more like the opposite. I am already very close, I am trying to modify the RGBLuminanceSource() luminance data but with no luck…I am not too sure what to apply mathematically to make it work.

    Would you happen to know how to do that so that this kind of qr can be scanned?
    https://dl.dropboxusercontent.com/u/17356625/Unity/projects/misc/test.png

    The annoying thing is that the same qr but negative ( using photoshop), will scan just fine…

    Thanks :)

    Jean

  5. ahmed

    hey dude i wanted to ask about if this method will still work with Unity3d fourth new edition ??? thanks in advace

  6. Hi, It’s really odd. I am on u3.5 and I have everything working almost, it detected the qr code fine but the code fails here in your behavior sample: if (data.BarcodeFormat == BarcodeFormat.QR_CODE) with an unity error: CompareBaseObjects can only be called from the main thread.
    Constructors and field initializers will be executed from the loading thread when loading a scene.
    Don’t use this function in the constructor or field initializers, instead move initialization code to the Awake or Start function.

    I am not good with threads… so maybe you have an idea of what could go wrong here? thanks a lot!

    Jean

    • whydoidoit

      Sorry just saw this (the site is supposed to be putting comments in my forum, so I’ve no idea why its ended up here – must have been a WordPress upgrade issue).

      If you haven’t managed to fix this – it would be down to using Loom.QueueOnMainThread before comparing the objects or comparing something to null.

      • Hi, we’re playing cat and mouse :) indeed I was able to solve the problem, by simply not relying on the Loom.QueueOnMainThread((). It simply didn’t work for me for some very odd reasons. so what I do is simply save the result in VuforiaScanner class and then check on a regular MonoBehaviour.update()

        thanks :)

  7. Abdullah Khalid

    what does it means by installed version of “Vuforia”…..i just import your package and then copy above class into “VuforiaScanner” and place it on GameObject…..Here is the error

    Assets/VuforiaScanner.cs(15,12): error CS0246: The type or namespace name `QCARBehaviour’ could not be found. Are you missing a using directive or an assembly reference?

    Can you please reply me all the steps for that

  8. ahmed

    Hello ,, first thanks u ALLLOOT u made the app alot easier to me ! ,,,, i used to port from Unity to Eclipse for the QR part ,,, but now should i create new scene for the QR scanner ,,, if so how to integrate the script to the scene ,,,and how to make like a framewith the size if the qr overlayed to the camera ??

    thanks
    Ahmed Tawheed

    • whydoidoit

      I just have a button to start the Vuforia scanner and I don’t have anything in the scene tracked so it just appears as a picture (with my NGUI toolbar at the top)

      • ahmed

        i want to make a reciever for scannedQRcode to display data as popup message like Toast in Android,,,, in the target object i made the script and added the function scannedQRcode as to listen to a reciever but i can’t call it as object is implicitly called in Unity c# i can’t call it in start() method i think

  9. Hiya,

    When I try and use the script above with the project, I get the following error

    error CS0246: The type or namespace name `QCARBehaviour’ could not be found. Are you missing a using directive or an assembly reference?

    Any ideas please?

    And thanks :)

  10. Shahram

    Thank you for your brilliant work. I managed to show the QR text easily on the GUI Text. Can you give me a hint that how I can show an AR Object on top of the QR code? Also is it possible for you to update your Unity friendly package with ZXing.net or advice me on how to do that as well?

    Thanks a lot again.

    Warmest Regards,
    Shahram

    • whydoidoit

      Unfortunately the AR tracking and the QR codes are two separate systems. I use a QR code to identify the application and then use standard Vurforia tracker markers to locate the scene in the 3D world. So the player scans a QR code and then points at a tracker marker. The QR code can be embedded in the marker.

      I’m not sure how you would incorporate ZXing.net yet – I was going to look at that but have become busy on other things. If you just use it as it is, it should provide all of the functionality of bar code reading and QR scanning – which is why I haven’t felt the need to change it yet.

  11. What kind of frame rate are you seeing with this?

    • whydoidoit

      When I’m scanning for QR codes it is a little hard to tell given that much of my geometry is disabled so I’m getting 30fps.

  12. Thank you for your brilliant work!!!
    How would you then send the read string to a GUIText?
    I’m trying to make it work on an iphone…

    Ciao!

    Giovanni

    • Thanks :)

      So if you use that script in the article the easiest way to do what you want is to set the “target” variable in the inspector to the object that contains your GUIText and then write a script with a ScannedQRCode function that you attach to that object.

      Something like this:

      public GUIText qrCodeLabel;

      void ScannedQRCode(string text)
      {
      qrCodeLabel.text = text;
      }

    • Or ScannedBarCode or both depending on what you want!

  13. Hi Mike,

    nice solution.
    I took your code and modified it so that it works with ZXing.Net.
    ZXing.Net is an up-to-date port of zxing. ZXing.Net was created and is maintained by me.
    Since version 0.7 ZXing.Net provides a Unity-friendly assembly.
    I couldn’t test the functionality of my version of your sample yet but it compiles successfully.
    You can find it here if you are interested:
    https://zxingnet.svn.codeplex.com/svn/trunk/Clients/VuforiaDemo
    If not or if it is not ok for you drop me an email and feel free to remove my comment.

    Regards,
    MIchael

Home Project With Code Unity, Vuforia & ZXing – Barcode scanning in AR games