Implement WitnessCalc in native apps Pt.1
Phat
Posted on November 21, 2024
In this short guide we will roughly implement generated witnesscalc .cpp files from circom
in react-native
app.
I'll do it in expo project.
Circom files will be borrowed from passport-zk-circuits
And finally, we will use 0xPolygonID/witnesscalc to compile executable
files for different native platforms.
Prerequisities:
By following this guide, let's assume you are already familiar with react-native
Android
Getting started
- install Android Studio
- open the settings and install
ndk v.23.1.7779620
- configure path in your
.bashrc
or.zshrc
config file
- clone https://github.com/rarimo/passport-zk-circuits, open project, and install all deps.
- follow readme to compile circuits
- we will test on auth.circom
- after compile we should get something similar to:
auth.cpp
andauth.dat
is the files we will usefollow readme "preparation" to install necessary deps
Now let's take a look at src
folder:
As you can see, for every [name].cpp
and [name].dat
files, there is also attached witnesscalc_[name].cpp
and witnesscalc_[name].h
files. And we need to do the same.
So, let's do it step-by-step:
- First things first: wrap content inside our auth.cpp file,
#include ...
#include ...
namespace CIRCUIT_NAME {
// millions of code lines here
} // namespace
remove #include and replace all "assert(" with "check(", that's will return exception onError in runtime, instead of crash
- Open the root
./CMakeLists.txt
file and scroll down to install(TARGETS...) and add the name of our files, e.g. "auth"
- create
witnesscalc_auth.cpp
andwitnesscalc_auth.h
files under src folder
You could check all the same files, their content is exact same, except the name, so we have done like so.
- now add these files to
./CMakeLists.txt
- And finally, open
./src/CMakeLists.txt
and add this code at the end of file
as before, you should notice, that this code is the same as code above in this file, so we just copied them and changed name
And these steps should be done for every .cpp & .dat files you will add.
in this case you can use this bash script
Also you could comment all unnecessary files in compile list, cuz it’s very time consuming process
in ./src/CMakeLists.txt
file
in root ./CMakeLists.txt
file
except fr
, it’s necessary
and also except src/witnesscalc.h
, it’s necessary too
Now you can run these commands to compile your executable files
./build_gmp.sh android // only once
make android // every time u want to compile new wtns calcs
The result:
libwitnesscalc_auth.so - is what we need for our android app, along with witnesscalc_auth.h file
Now we can jump to our react-native app.
note, that in this guide i'm using new architecture, so you could follow this guide to turn it on.
We will implement our generated file in native-module
And use https://github.com/callstack/react-native-builder-bob for that.
so let's start
npx create-react-native-library@latest wtnscalcs
After that, run yarn install
, rebuild app
, then run android, and we will be able to use this module, cuz there is already been example methods generated by “builder-bob”
npx expo prebuild --platform android --clean && npx expo run:android
the usage is simple, as usual npm-package
import { multiply } from 'rn-wtnscalcs'
console.log(multiply(2, 3))
Now, let’s dive deep into our module
this is what we see after scaffolding our module, and to modify the code, it’s would be better to open via android-studio
we ARE NOT going to open this android folder, but should open a root android folder, this one:
and after opening it in android-studio and waiting for indexing we will see:
here is our module, and by modifying it, our code in ./modules/[module]/android folder will be changed too
so what we see now:
The entry for us would be build.gradle file of our module:
and the cmake structure which points to our CMakeLists.txt, which is register our future cpp file with logic
next, the rn-wtnscalcs.cpp file, where common logic are placed
and then .h file for it
The bridging file, which bridging cpp code to java/kotlin code
Now, let’s back to our "frontend perspective" e.g IDE or editor where your react-native project are, and take a look in to spec file in module, which responsible to register our module with native code and define top-level interface
By defining methods in this "Spec" interface
and after run
npx expo prebuild --platform android --clean && npx expo run:android
we will generate abstract class in our android project
And after that we could implement this class in main module .kt file
And by exposing method from index.tsx
we will be able to call method from module as a regular npm-pkg
Okay, let's add new "plus" method just for practice:
import { multiply, plus } from 'rn-wtnscalcs'
console.log(multiply(2, 3))
console.log(plus(2, 3))
Nice, now let's jump straight in to hard mode:
let's define what we need:
1) first of all, at the very top level we have "inputs" which witness calculator will accept in the future
2) we need to send these inputs to our native module, pass it to .so file
3) execute and return bytes, it should be the same as circom compiles .wtns file, but we will keep file content in ram, instead of file
generate spec file
npx expo prebuild --platform android --clean && npx expo run:android
Now, let's write this implementation code, and leave it empty for a while
Now, the most important part - we will add our .so library in to project
- Add new raw directory in our res folder
and paste there our auth.dat file
- add our witnesscalc_auth.h file to cpp folder
- get back to "Frontend perspective", create lib folder under modules/android directory and paste there a
.so
file
- and finally register our library in CMakeLists.txt file
cmake_minimum_required(VERSION 3.4.1)
project(RnWtnscalcs)
set (CMAKE_VERBOSE_MAKEFILE ON)
set (CMAKE_CXX_STANDARD 14)
add_library(rn-wtnscalcs SHARED
../cpp/rn-wtnscalcs.cpp
cpp-adapter.cpp
)
# Specifies a path to native header files.
include_directories(
../cpp
)
link_directories(lib)
add_library(auth SHARED IMPORTED)
set_target_properties(auth PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/lib/libwitnesscalc_auth.so)
target_link_libraries(rn-wtnscalcs
android
auth)
what we have done:
- We linked directories where we've placed our .so file
- Registered our .so file as library
- And finally ‘target_link_libraries’ will compile it for us in one executable file, which we can call in runtime
After these steps we will be able to execute .so file in runtime
let's write the code to call it:
object WtnsUtil {
external fun auth(
circuitBuffer: ByteArray?,
circuitSize: Long,
jsonBuffer: ByteArray?,
jsonSize: Long,
wtnsBuffer: ByteArray?,
wtnsSize: LongArray?,
errorMsg: ByteArray?,
errorMsgMaxSize: Long
): Int
init {
System.loadLibrary("rn-wtnscalcs")
}
}
define our object with external method of our auth witness calculator
and init
will load our .so library, which we have defined in CMakeLists.txt
Now, we could open options for our auth method, cuz there is no binded code for external function
these steps will create binded function in our rn-wtnscalcs.cpp file, but let’s move it to cpp-adapter.cpp
that’s the place where all external function will be called from, and now let’s just add this code inside as shown above:
const char *circuitBuffer = reinterpret_cast<const char *>(env->GetByteArrayElements(
circuit_buffer, nullptr));
const char *jsonBuffer = reinterpret_cast<const char *>(env->GetByteArrayElements(json_buffer,
nullptr));
char *wtnsBuffer = reinterpret_cast<char *>(env->GetByteArrayElements(wtns_buffer, nullptr));
char *errorMsg = reinterpret_cast<char *>(env->GetByteArrayElements(error_msg, nullptr));
unsigned long wtnsSize = env->GetLongArrayElements(wtns_size, nullptr)[0];
int result = witnesscalc_auth(
circuitBuffer, static_cast<unsigned long>(circuit_size),
jsonBuffer, static_cast<unsigned long>(json_size),
wtnsBuffer, &wtnsSize,
errorMsg, static_cast<unsigned long>(error_msg_max_size));
// Set the result and release the resources
env->SetLongArrayRegion(wtns_size, 0, 1, reinterpret_cast<jlong *>(&wtnsSize));
env->ReleaseByteArrayElements(circuit_buffer,
reinterpret_cast<jbyte *>(const_cast<char *>(circuitBuffer)), 0);
env->ReleaseByteArrayElements(json_buffer,
reinterpret_cast<jbyte *>(const_cast<char *>(jsonBuffer)), 0);
env->ReleaseByteArrayElements(wtns_buffer, reinterpret_cast<jbyte *>(wtnsBuffer), 0);
env->ReleaseByteArrayElements(error_msg, reinterpret_cast<jbyte *>(errorMsg), 0);
return result;
Now, lets create a class next to WtnsUtil
object, which will accept our external function as a parameter, allocate memory and execute calculation, almost it should get access to our dat file, so we need to pass AssetManager and context:
class WtnsCalculator(val context: Context, val assetManager: AssetManager) {
fun calculateWtns(
datFile: Int,
inputs: ByteArray,
wtnsCalcFunction: (
circuitBuffer: ByteArray,
circuitSize: Long,
jsonBuffer: ByteArray,
jsonSize: Long,
wtnsBuffer: ByteArray,
wtnsSize: LongArray,
errorMsg: ByteArray,
errorMsgMaxSize: Long
) -> Int
): ByteArray {
val DFile = openRawResourceAsByteArray(datFile)
val msg = ByteArray(256)
val witnessLen = LongArray(1)
witnessLen[0] = 100 * 1024 * 1024
val byteArr = ByteArray(100 * 1024 * 1024)
val res = wtnsCalcFunction(
DFile,
DFile.size.toLong(),
inputs,
inputs.size.toLong(),
byteArr,
witnessLen,
msg,
256
)
if (res == 2) {
throw Exception("Not enough memory for wtns calculation")
}
if (res == 1) {
throw Exception("Error during wtns calculation ${msg.decodeToString()}")
}
val witnessData = byteArr.copyOfRange(0, witnessLen[0].toInt())
return witnessData
}
private fun openRawResourceAsByteArray(resourceName: Int): ByteArray {
val inputStream = context.resources.openRawResource(resourceName)
val byteArrayOutputStream = ByteArrayOutputStream()
try {
val buffer = ByteArray(1024)
var length: Int
while (inputStream.read(buffer).also { length = it } != -1) {
byteArrayOutputStream.write(buffer, 0, length)
}
return byteArrayOutputStream.toByteArray()
} finally {
// Close the streams in a finally block to ensure they are closed even if an exception occurs
byteArrayOutputStream.close()
inputStream.close()
}
}
}
and use it in implemented
function in our module.kt file, which we have leaved empty:
@RequiresApi(Build.VERSION_CODES.O)
override fun generateAuthWtns(jsonInputsBase64: String, promise: Promise) {
val wtnsCalc = WtnsCalculator(
reactApplicationContext,
reactApplicationContext.assets
)
try {
val res = wtnsCalc.calculateWtns(
R.raw.auth,
Base64.getDecoder().decode(jsonInputsBase64),
WtnsUtil::auth
).let {
// base64
String(Base64.getEncoder().encode(it))
}
promise.resolve(res)
} catch (e: Exception) {
promise.reject(e)
}
}
now, rebuild the project and let's test it:
npx expo prebuild --platform android --clean && npx expo run:android
import { Buffer } from 'buffer'
import { generateAuthWtns } from 'rn-wtnscalcs'
const runAuthCalc = async () => {
try {
const res = await generateAuthWtns(Buffer.from(JSON.stringify(authInputs)).toString('base64'))
console.log(Buffer.from(res, 'base64'))
} catch (error) {
ErrorHandler.processWithoutFeedback(error)
}
}
result:
Part 2 (ios)
Posted on November 21, 2024
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.