Implement WitnessCalc in native apps Pt.1

lukachi

Phat

Posted on November 21, 2024

Implement WitnessCalc in native apps Pt.1

In this short guide we will roughly implement generated witnesscalc .cpp files from circom in react-native app.

I'll do it in expo project.

Circom files will be borrowed from passport-zk-circuits

And finally, we will use 0xPolygonID/witnesscalc to compile executable files for different native platforms.

Prerequisities:
By following this guide, let's assume you are already familiar with react-native


Android

Getting started

  • install Android Studio
  • open the settings and install ndk v.23.1.7779620

Image description

  • configure path in your .bashrc or .zshrc config file Image description

Now let's take a look at src folder:

Image description

As you can see, for every [name].cpp and [name].dat files, there is also attached witnesscalc_[name].cpp and witnesscalc_[name].h files. And we need to do the same.

So, let's do it step-by-step:

  1. First things first: wrap content inside our auth.cpp file,
#include ... 
#include ... 

namespace CIRCUIT_NAME {

// millions of code lines here

} // namespace
Enter fullscreen mode Exit fullscreen mode

remove #include and replace all "assert(" with "check(", that's will return exception onError in runtime, instead of crash

  1. Open the root ./CMakeLists.txt file and scroll down to install(TARGETS...) and add the name of our files, e.g. "auth"

Image description

  1. create witnesscalc_auth.cpp and witnesscalc_auth.h files under src folder

Image description

Image description

You could check all the same files, their content is exact same, except the name, so we have done like so.

  1. now add these files to ./CMakeLists.txt

Image description

  1. And finally, open ./src/CMakeLists.txt and add this code at the end of file

Image description

as before, you should notice, that this code is the same as code above in this file, so we just copied them and changed name

And these steps should be done for every .cpp & .dat files you will add.

in this case you can use this bash script

Also you could comment all unnecessary files in compile list, cuz it’s very time consuming process

in ./src/CMakeLists.txt file

Image description

in root ./CMakeLists.txt file

Image description

except fr, it’s necessary

Image description

and also except src/witnesscalc.h, it’s necessary too

Now you can run these commands to compile your executable files

./build_gmp.sh android // only once
make android // every time u want to compile new wtns calcs
Enter fullscreen mode Exit fullscreen mode

The result:

Image description

libwitnesscalc_auth.so - is what we need for our android app, along with witnesscalc_auth.h file


Now we can jump to our react-native app.

note, that in this guide i'm using new architecture, so you could follow this guide to turn it on.

We will implement our generated file in native-module

And use https://github.com/callstack/react-native-builder-bob for that.

so let's start

npx create-react-native-library@latest wtnscalcs
Enter fullscreen mode Exit fullscreen mode

Image description

Image description

Image description

After that, run yarn install, rebuild app, then run android, and we will be able to use this module, cuz there is already been example methods generated by “builder-bob”

npx expo prebuild --platform android --clean && npx expo run:android 
Enter fullscreen mode Exit fullscreen mode

the usage is simple, as usual npm-package

import { multiply } from 'rn-wtnscalcs'

console.log(multiply(2, 3))
Enter fullscreen mode Exit fullscreen mode

Now, let’s dive deep into our module

Image description

this is what we see after scaffolding our module, and to modify the code, it’s would be better to open via android-studio

we ARE NOT going to open this android folder, but should open a root android folder, this one:

Image description

and after opening it in android-studio and waiting for indexing we will see:

Image description

here is our module, and by modifying it, our code in ./modules/[module]/android folder will be changed too

so what we see now:

Image description

The entry for us would be build.gradle file of our module:

Image description

and the cmake structure which points to our CMakeLists.txt, which is register our future cpp file with logic

Image description

next, the rn-wtnscalcs.cpp file, where common logic are placed

Image description

and then .h file for it

Image description

The bridging file, which bridging cpp code to java/kotlin code

Image description


Now, let’s back to our "frontend perspective" e.g IDE or editor where your react-native project are, and take a look in to spec file in module, which responsible to register our module with native code and define top-level interface

By defining methods in this "Spec" interface

Image description

and after run

npx expo prebuild --platform android --clean && npx expo run:android

we will generate abstract class in our android project

Image description

Image description

And after that we could implement this class in main module .kt file

Image description

And by exposing method from index.tsx we will be able to call method from module as a regular npm-pkg

Image description


Okay, let's add new "plus" method just for practice:

Image description

Image description

Image description

Image description

Image description

Image description

import { multiply, plus } from 'rn-wtnscalcs'

console.log(multiply(2, 3))
console.log(plus(2, 3))
Enter fullscreen mode Exit fullscreen mode

Image description


Nice, now let's jump straight in to hard mode:

let's define what we need:
1) first of all, at the very top level we have "inputs" which witness calculator will accept in the future
2) we need to send these inputs to our native module, pass it to .so file
3) execute and return bytes, it should be the same as circom compiles .wtns file, but we will keep file content in ram, instead of file

Image description

generate spec file

npx expo prebuild --platform android --clean && npx expo run:android
Enter fullscreen mode Exit fullscreen mode

Image description

Now, let's write this implementation code, and leave it empty for a while

Image description

Now, the most important part - we will add our .so library in to project

  1. Add new raw directory in our res folder

Image description

and paste there our auth.dat file

Image description

  1. add our witnesscalc_auth.h file to cpp folder

Image description

  1. get back to "Frontend perspective", create lib folder under modules/android directory and paste there a .so file

Image description

  1. and finally register our library in CMakeLists.txt file

Image description

cmake_minimum_required(VERSION 3.4.1)
project(RnWtnscalcs)

set (CMAKE_VERBOSE_MAKEFILE ON)
set (CMAKE_CXX_STANDARD 14)

add_library(rn-wtnscalcs            SHARED
            ../cpp/rn-wtnscalcs.cpp
            cpp-adapter.cpp
)

# Specifies a path to native header files.
include_directories(
            ../cpp
)

link_directories(lib)

add_library(auth SHARED IMPORTED)
set_target_properties(auth PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/lib/libwitnesscalc_auth.so)

target_link_libraries(rn-wtnscalcs
        android
        auth)

Enter fullscreen mode Exit fullscreen mode

what we have done:

  1. We linked directories where we've placed our .so file
  2. Registered our .so file as library
  3. And finally ‘target_link_libraries’ will compile it for us in one executable file, which we can call in runtime

After these steps we will be able to execute .so file in runtime

let's write the code to call it:

Image description

object WtnsUtil {
  external fun auth(
    circuitBuffer: ByteArray?,
    circuitSize: Long,
    jsonBuffer: ByteArray?,
    jsonSize: Long,
    wtnsBuffer: ByteArray?,
    wtnsSize: LongArray?,
    errorMsg: ByteArray?,
    errorMsgMaxSize: Long
  ): Int

  init {
    System.loadLibrary("rn-wtnscalcs")
  }
}
Enter fullscreen mode Exit fullscreen mode

define our object with external method of our auth witness calculator

and init will load our .so library, which we have defined in CMakeLists.txt

Now, we could open options for our auth method, cuz there is no binded code for external function

Image description

Image description

Image description

these steps will create binded function in our rn-wtnscalcs.cpp file, but let’s move it to cpp-adapter.cpp

Image description

that’s the place where all external function will be called from, and now let’s just add this code inside as shown above:

    const char *circuitBuffer = reinterpret_cast<const char *>(env->GetByteArrayElements(
            circuit_buffer, nullptr));
    const char *jsonBuffer = reinterpret_cast<const char *>(env->GetByteArrayElements(json_buffer,
                                                                                      nullptr));
    char *wtnsBuffer = reinterpret_cast<char *>(env->GetByteArrayElements(wtns_buffer, nullptr));
    char *errorMsg = reinterpret_cast<char *>(env->GetByteArrayElements(error_msg, nullptr));

    unsigned long wtnsSize = env->GetLongArrayElements(wtns_size, nullptr)[0];


    int result = witnesscalc_auth(
            circuitBuffer, static_cast<unsigned long>(circuit_size),
            jsonBuffer, static_cast<unsigned long>(json_size),
            wtnsBuffer, &wtnsSize,
            errorMsg, static_cast<unsigned long>(error_msg_max_size));

    // Set the result and release the resources
    env->SetLongArrayRegion(wtns_size, 0, 1, reinterpret_cast<jlong *>(&wtnsSize));

    env->ReleaseByteArrayElements(circuit_buffer,
                                  reinterpret_cast<jbyte *>(const_cast<char *>(circuitBuffer)), 0);
    env->ReleaseByteArrayElements(json_buffer,
                                  reinterpret_cast<jbyte *>(const_cast<char *>(jsonBuffer)), 0);
    env->ReleaseByteArrayElements(wtns_buffer, reinterpret_cast<jbyte *>(wtnsBuffer), 0);
    env->ReleaseByteArrayElements(error_msg, reinterpret_cast<jbyte *>(errorMsg), 0);

    return result;
Enter fullscreen mode Exit fullscreen mode

Now, lets create a class next to WtnsUtil object, which will accept our external function as a parameter, allocate memory and execute calculation, almost it should get access to our dat file, so we need to pass AssetManager and context:

class WtnsCalculator(val context: Context, val assetManager: AssetManager) {
  fun calculateWtns(
    datFile: Int,
    inputs: ByteArray,
    wtnsCalcFunction: (
      circuitBuffer: ByteArray,
      circuitSize: Long,
      jsonBuffer: ByteArray,
      jsonSize: Long,
      wtnsBuffer: ByteArray,
      wtnsSize: LongArray,
      errorMsg: ByteArray,
      errorMsgMaxSize: Long
    ) -> Int
  ): ByteArray {
    val DFile = openRawResourceAsByteArray(datFile)

    val msg = ByteArray(256)

    val witnessLen = LongArray(1)
    witnessLen[0] = 100 * 1024 * 1024

    val byteArr = ByteArray(100 * 1024 * 1024)

    val res = wtnsCalcFunction(
      DFile,
      DFile.size.toLong(),
      inputs,
      inputs.size.toLong(),
      byteArr,
      witnessLen,
      msg,
      256
    )

    if (res == 2) {
      throw Exception("Not enough memory for wtns calculation")
    }

    if (res == 1) {
      throw Exception("Error during wtns calculation ${msg.decodeToString()}")
    }

    val witnessData = byteArr.copyOfRange(0, witnessLen[0].toInt())

    return witnessData
  }

  private fun openRawResourceAsByteArray(resourceName: Int): ByteArray {
    val inputStream = context.resources.openRawResource(resourceName)
    val byteArrayOutputStream = ByteArrayOutputStream()

    try {
      val buffer = ByteArray(1024)
      var length: Int

      while (inputStream.read(buffer).also { length = it } != -1) {
        byteArrayOutputStream.write(buffer, 0, length)
      }

      return byteArrayOutputStream.toByteArray()
    } finally {
      // Close the streams in a finally block to ensure they are closed even if an exception occurs
      byteArrayOutputStream.close()
      inputStream.close()
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Image description

and use it in implemented function in our module.kt file, which we have leaved empty:

Image description

@RequiresApi(Build.VERSION_CODES.O)
  override fun generateAuthWtns(jsonInputsBase64: String, promise: Promise) {
    val wtnsCalc = WtnsCalculator(
      reactApplicationContext,
      reactApplicationContext.assets
    )

    try {
      val res = wtnsCalc.calculateWtns(
        R.raw.auth,
        Base64.getDecoder().decode(jsonInputsBase64),
        WtnsUtil::auth
      ).let {
        // base64
        String(Base64.getEncoder().encode(it))
      }

      promise.resolve(res)
    } catch (e: Exception) {
      promise.reject(e)
    }
  }
Enter fullscreen mode Exit fullscreen mode

now, rebuild the project and let's test it:

npx expo prebuild --platform android --clean && npx expo run:android
Enter fullscreen mode Exit fullscreen mode
import { Buffer } from 'buffer'
import { generateAuthWtns } from 'rn-wtnscalcs'

const runAuthCalc = async () => {
    try {
      const res = await generateAuthWtns(Buffer.from(JSON.stringify(authInputs)).toString('base64'))
      console.log(Buffer.from(res, 'base64'))
    } catch (error) {
      ErrorHandler.processWithoutFeedback(error)
    }
  }
Enter fullscreen mode Exit fullscreen mode

result:

Image description

Part 2 (ios)

💖 💪 🙅 🚩
lukachi
Phat

Posted on November 21, 2024

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Implement WitnessCalc in native apps Pt.2
reactnative Implement WitnessCalc in native apps Pt.2

November 21, 2024

Implement WitnessCalc in native apps Pt.1
reactnative Implement WitnessCalc in native apps Pt.1

November 21, 2024