Adding Live eSign to Your WebRTC Application with Co-browsing and PDF Editing – WebRTC.ventures

Written by Tahir Golge | Oct 11, 2023 4:00:00 AM

Last week, my colleague Hector Zelaya authored a post called, Live eSign Saves a Trip to the Bank. He spoke about how the power trio of secure WebRTC video conferencing, co-browsing, and PDF editing could make visiting a financial institution or similar office to sign documents or provide visual proof of identity in front of an attorney, bank officer, lawyer, or notary a thing of the past.

Today, I’ll provide the technical details on exactly how to enable Live eSign in your WebRTC application.

Live eSign Prerequisites

It is assumed that you already have a secure video conferencing application. (Need to build one? We can help!)

The code examples here are written in TypeScript and run on NodeJS and Next.js. If your video conferencing application uses a different stack, you can still apply the concepts but you will need to adapt the code.

You will need a Docker and Docker Compose for running the co-browsing service locally.

A PubNub account is used to handle the events related with editing and sharing the PDF file.

Part One: Enabling PDF Editing

The first step is to let the employee from the institution (we call them the “power user”) add a PDF document to the application. To do so, let’s create a new React component to include in the video call page of your application.

In this new component we will perform three tasks:

Add a PDF document
Allow for PDF editing
Once finished with editing, render it within the co-browsing session

Adding a PDF File to the Application

Let’s start by adding the new ESignContainer component in your video call component, along with a useState hook for storing the URL of the PDF file. Such an URL is passed as a prop to the new component. Depending on the logic of your application you might need to pass additional props to it.

...
// the video call component
export const MeetingContainer = function MeetingContainer() {
  ...
  // state hook for file URL
  const [fileUrl, setFileUrl] = useState('')
  ...
  // add the new container and pass fileUrl as prop
  return (
    ...
    <ESignContainer
      ...
      fileUrl={fileUrl} />
  )
}

Now we need a way to let all participants know when there is a file available. For this, we will subscribe the participants to a PubNub channel. We need to set a unique name for the channel, perhaps the identifier of the video call session or room name. Make sure to adapt this to your application.

We then use the useEffect hook to initialize the PubNub client and start listening for new messages. For now, we just want to set the file URL in the state. In this post, we use the pubnub-react library to initialize the client through a convenient hook.

// import pubnub-react library
import { usePubNub } from 'pubnub-react'
...
// the video call component
const const MeetingContainer = function MeetingContainer(){
  ...
  // initialize PubNub client
  const pubnub = usePubNub()
  // set the channel name to room name
  // or whatever unique value works for your application
  const channel = ['roomName'] 
  ...
  useEffect(() => {
    // variable for listener
    let listenerParams

    // different subscriptions types for users and power users
    if (user.role === 'client') {
      listenerParams = {
        // when a file arrives, store the file's URL in the state
        file(event) {
          setFile(event.file.url)
        } 
      }
    } else {
      listenerParams = {
        // same for power user
        file(event) {
          setFile(event.file.url)
        }
      }
    }

    // initialize the listener
    pubnub.addListener(listenerParams)
    pubnub.subscribe({ channel })
    return () => {
      // deregister the listener when unmounting
      pubnub.unsubscribe({ channel })
      pubnub.removeListener(listenerParams)
    }
  }, [pubnub, channel])
  ...
}

Now, let’s create this ESignContainer component. Here we want to manage the files that are shared and the state of such shares. This includes React’s useState hooks for such files and a set of multiple flags that indicates their state.

It also includes handler functions for when power users drag and drop files or when they click the “Add” button. In addition, we use PubNub’s sendFile feature to set up temporary storage for the PDF file and also to share its URL.

We also create a new component, ESignView, which allows power users to add the PDF file and render it when it’s available. We will see this component in depth later.

...
export const ESignContainer = function ESignContainer({
	...
	fileUrl // we pass the fileUrl as props, along with any other
              // that your application may need
}: Props) {
  // useState hook for files
  const [files, setFiles] = useState([])
  // a flag that indicates if file has been uploaded to PubNub
  const [uploadFinished, setUploadFinished] = useState(false)
  // a flag that indicates if the power user has clicked the Add button
  const [uploadClicked, setUploadClicked] = useState(false)
  ...
  // handler function for when power users drag and drop files
  const handleDrop = (e) => {
    e.preventDefault()
    const filesArray = []
    const { items } = e.dataTransfer
    for (let i = 0; i < items.length; i++) {
      if (items[i].kind === 'file') {
        const file = items[i].getAsFile()
        filesArray.push(file)
      }
    }
    setFiles(filesArray)
  }
  // handler function for when power users click the Add button
  const handleUpload = async () => {
    files.forEach(async (file) => {
      // we upload the file to PubNub and notify users
      const result = await pubnub.sendFile({
        channel: 'roomName',
        file,
        message: 'AdminFile',
      })
    })
  }
  ...
  // rendering the UI component for PDF upload and editing
  // if the upload hasn't finished, we show the ESignView component
  // which we will look later
  return (
    ...
    {!uploadFinished && user.role === 'powerUser' && (
      <ESignView
        files={files}
        fileUrl={fileUrl}
        handleDrop={handleDrop}
        handleUpload={handleUpload}
        uploadClicked={uploadClicked}
        setUploadClicked={setUploadClicked} />
    )}
  )
}

Now let’s take a look at the ESignView component. Here we want to show a form where the power user can add a PDF file. After that, we want to enable the power user to prepare the file before sharing it with the client.

To do so, we render a simple upload form with support for dragging and dropping files. This is done by setting the handleDrop function we defined before in its onDrop attribute. The form includes a button that initiates the upload through the, previously defined, handleUpload function that is passed to its onClick attribute.

The PDF editing capabilities will come from the PSPDFKit library, which we’ll explore in the next section. For now, we simply add an HTML div tag where the PDF editor will live.

...
export const ESignView = function ESignView({
  // props
}: Props) {
  return (
    <div
      {/* we add the onDrop and onDragOver events */}
      onDrop={handleDrop}
      onDragOver={(e) => e.preventDefault()}
    >
      {/* we show the form until the user clicks Add */}
      {!uploadClicked && (
        <div>
          <p>Drag and drop file here and click to upload</p>
          <small>Formats accepted: pdf, doc, docx</small>
          <ul>
            {files.map((file) => (
              <li key={file.name}>{file.name}</li>
            ))}
          </ul>
          <Button type="button" onClick={handleUpload}>
            Add
          </Button>
        </div>
      )}
      {/* div for PDF editor */}
      <div />
    </div>
  )
}

Enter PSPDFKit

Once the file is available in PubNub, we download and render it so the power user can edit it before sharing it with the client. We will use the PSPDFKit library. As per the library documentation, let’s start by installing the library and copying the web assets to public folder of the project, as follows:

npm install pspdfkit
cp -R ./node_modules/pspdfkit/dist/pspdfkit-lib public/pspdfkit-lib

Next, we need to actually render a file. The code that we wrote so far allows a power user to upload the file to PubNub. When this happens, PubNub will let all participants know that there is a file available, they, in turn, store its URL in the fileUrl variable from the ESignContainer component. Let’s use that fileUrl variable to know when the file is ready to edit and actually render it using PSPDFKit.

First, let’s use a useState hook to store an instance of the PDF editor. We will use this instance to export the final PDF file. We also need to initialize the PSPDFKit client and also use the useRef hook to get the reference of the HTML div tag where the editor renders.

The PDF magic happens inside a useEffect hook that reacts to fileUrl value changes. The first thing we do here is download the file, using the axios library, and add it to the editor. For this, we add a custom convertPdfToBase64 function that takes the fileUrl and converts it into something usable by the editor.

...
export const ESignView = function ESignView({
  ...
  // useState hook for the PDF editor instance
  const [instance, setInstance] = useState<any>()
  // PSPDFKit variable
  let PSPDFKit
  // useRef hook for the div element
  const containerRef = useRef(null)

  // custom function for converting PDF up base64
  const convertPdfToBase64 = async (url) => {
    try {
      // Download the PDF file
      const response = await axios.get(url, {
        responseType: 'arraybuffer',
      })
      const pdfData = Buffer.from(response.data, 'binary')

      // Convert PDF data to Base64
      const base64Data = pdfData.toString('base64')
      return base64Data
  } catch (error) {
      console.error('Error converting PDF to Base64:', error)
      return null
    }
  }

  // all the magic happens here
  useEffect(() => {
    const container = containerRef.current

    ;(async function () {
      PSPDFKit = await import('pspdfkit')

      if (PSPDFKit) {
        PSPDFKit.unload(container)
      }
      if (fileUrl) {
        setUploadClicked(true)
        convertPdfToBase64(fileUrl).then(async (base64Data) => {
          await PSPDFKit.load({
            container,
            document: `data:application/pdf;base64,${base64Data}`,
            baseUrl: `${window.location.protocol}//${window.location.host}/`,
            toolbarItems: [
              ...PSPDFKit.defaultToolbarItems,
              {
                type: 'form-creator',
              },
            ],
            initialViewState: new PSPDFKit.ViewState({
              interactionMode: 
                PSPDFKit.InteractionMode.FORM_CREATOR,
            }),
          }).then((ins) => {
            console.log('Succesfully loaded')
            setInstance(ins)
          })
        })
      }
    })()

    return () => PSPDFKit && PSPDFKit.unload(container)
  }, [fileUrl])
  …
  {/* Adding ref attribute to div for PDF editor */}
  <div ref={containerRef} />
}

Start PDF Sharing

Your application should now have the ability to upload a PDF file, store it in PubNub, and allow the power user to edit it. Now, let’s add a way to share the file with the rest of the participants.

For this, we include a new flag in the ESignContainer: shareAndStart. This flag indicates that the file is ready and the participants can start with eSigning. We also have a new useState hook, finalPDFBuffer, that stores the final PDF before it’s added to PubNub.

The new flag is enabled by clicking a new “Share Document and Start eSign” button that is available while editing the file and after clicking the upload button. The value of such flag is passed to the ESignView component, we will look at this later.

We also listen for finalPDFBuffer value changes, and when the file is ready we upload it to PubNub, again using sendFile feature. After that, we’re ready for co-browsing. More on that in the next section.

Besides the new flag and hook, we also want to send a message to the PubNub channel that notifies all participants that they can join the co-browsing session once it’s ready. For now, we will define a sendShareAndStart function that will get called when clicking the button mentioned above.

export const ESignContainer = function EsignContainer({ // props }) {
  ...
  // new flag for sharing pdf file and starting esign
  const [shareAndStart, setShareAndStart] = useState(false)
  // new variable for storing final PDF file as buffer
  const [finalPDFBuffer, setFinalPDFBuffer] = useState()
  ...
  // function for sending the message to PubNub channel
  const sendStartEsign = async () => {
    const publishPayload = {
      channel: 'roomName',
      message: {
        title: 'StartEsign'
      },
    }
    // Send the notification
    const resp = await pubnub.publish(publishPayload)
  }

  // function for sending the final PDF to PubNub channel
  const handleUploadComplete = async () => {
    if (finalPDFBuffer) {
      const uint8Array = new Uint8Array(finalPDFBuffer)

      // Create an object with the Uint8Array data
      const fileObject = {
        data: uint8Array,
        name: 'file.pdf',
        type: 'application/pdf',
      }

      const result = await pubnub.sendFile({
        channel: 'roomName',
        file: fileObject,
        message: 'FinalFile',
      })

      setUploadFinished(true)
    }
  }

  // we listen for changes in finalPDFBuffer
  useEffect(() => {
    handleUploadComplete()
  }, [finalPDFBuffer])
  ...
  return (
    ...
    {uploadClicked && user.role === 'powerUser' && (
      <Button
        variant="primary"
        onClick={async () => {
            // send the URL
            await sendStartEsign()
            // set the flag as true
            setShareAndStart(true)
          }
        }
      >
        Share Document and Start eSign
      </Button>
    )}
    ...
      <ESignView
        ...
        setFinalPDFBuffer={setFinalPDFBuffer}
        shareAndStart={shareAndStart} />
    ...
  )
}

Now, let’s take a look at the ESignView component. Here, we listen for changes in the shareAndStart variable that we receive as prop, and when an instance of the PDF file is ready, we assign it to the finalPDFBuffer state we just created. This logic lives in a separate function and we use the useCallback hook to make it optimal across component renders. Another thing we do is render the PDF file only if the shareAndStart flag is false, which is the case for when editing the file.

export const ESignView = function ESignView({
  ...
  setFinalPDFBuffer,
  shareAndStart
} : Props) {
  ...
  // a function to set finalPDFBuffer variable when an instance of PDF
  // file is available
  const getArrayBuffer = useCallback(async () => {
    if (instance) {
      const buffer = await instance.exportPDF()
      setFinalPDFBuffer(buffer)
    }
  }, [PSPDFKit, instance])

  // we listen for changes in shareAndStart prop
  useEffect(() => {
    getArrayBuffer()
  }, [shareAndStart])

  useEffect(() => {
    if (!shareAndStart) {
      // the code that renders the PDF goes here
    }
  }, [fileUrl])
  ...
}

Now we’re ready for co-browsing! But before that, let’s create a component for the page we will co-browse to. We will call this component ESignPageContainer. In this new component, we use PSPDFKit to render the more recent PDF file that has been uploaded to the PubNub channel.
We also need the unique identifier for the channel, which we will call esignId. In our example we have set this to the room name, make sure you adapt this to the logic of your application.

export const ESignPageContainer = function ESignPageContainer() {
  // useState hook for storing PDF file's URL
  const [fileUrl, setFileUrl] = useState<string>()

  // hooks and variables for PSPDFKit
  const [instance, setInstance] = useState<any>()
  let PSPDFKit
  const containerRef = useRef(null)

  // usePubNub hook for initializing PubNub variable
  const pubnub = usePubNub()

  // set the esignId 
  const esignId = roomName // or any other value that makes sense for 
                           // your application

  // useEffect hook for downloading the more recent file from PubNub
  useEffect(() => {
    // list all the available files in the channel
    pubnub.listFiles(
      {
        channel: esignId,
        limit: 25,
      },
      (status, response) => {
        // sort the files from newest to oldest
        const sortedFiles = response.data.sort(
          (a, b) =>
            new Date(b.created).getTime() -
            new Date(a.created).getTime()
        )
        // get the URL of the file
        const result = pubnub.getFileUrl({
          channel: esignId,
          id: sortedFiles[0].id,
          name: sortedFiles[0].name,
        })
        // set the file URL in the state
        setFileUrl(result)
      }
    )
  }, [])

  // useEffect hook for rendering PDF editor
  useEffect(() => {
    // PSPDFKit code goes here
  }, [fileUrl])

  return (
    <div>
      <div
        ref={containerRef}
        style={{ height: '100%', width: '100%' }}
      />
    </div>
  )
}

Part Two: Integrating Co-Browsing Functionality

Co-browsing is enabled using n.eko. n.eko defines itself as, “a self hosted virtual browser that runs in Docker.” This tool allows you to run a virtual browser in a remote server that you control via WebRTC. It’s an ideal solution for developers to test web applications, privacy-conscious users that seek for a secure browsing experience, or anything that can take advantage of a virtual server running in an isolated environment.

For our particular use case, the most interesting feature is that n.eko allows multiple users to access the browser simultaneously. Hence, it provides that “shared” context where Live eSign takes place.

Running n.eko Locally

The first step is to have n.eko running. This can be done easily using Docker and Docker Compose. The official documentation provides an example of a docker-compose.yml file that uses Google Chrome. Add this file to your project.

version: "3.4"
services:
  neko:
    image: "m1k1o/neko:google-chrome"
    restart: "unless-stopped"
    shm_size: "2gb"
    ports:
      - "8080:8080"
      - "52000-52100:52000-52100/udp"
    cap_add:
      - SYS_ADMIN
    environment:
      NEKO_SCREEN: '1920x1080@30'
      NEKO_PASSWORD: neko
      NEKO_PASSWORD_ADMIN: admin
      NEKO_EPR: 52000-52100
      NEKO_NAT1TO1: 127.0.0.1

Then you can run n.eko using the command docker-compose up. As a result you will have a web client available at http://localhost:8080 that you can use to interact with the remote browser via WebRTC.

Adding n.eko to a Video Conferencing Application

The easiest way to add this functionality to your video conferencing application is via an HTML iframe tag that shows n.eko’s web client. You can even build your own client so it behaves the way you want and add additional features, such as a more advanced authentication mechanism or supporting extra parameters. For this post we will use the one that comes by default.

Before looking into this, let’s take it from where we left it in the previous section. As you may recall, after the power user clicks the “Share Document and Start eSign” button, the application notifies clients that it’s ready to start eSign, and also updates the shareAndStart flag, which in turn triggers a function that sets the finalPDFBuffer. Once the finalPDFBuffer is ready, it’s uploaded to PubNub using sendFile.

As a result, there are two messages that we need to listen for: one is the StartEsign message that lets clients know that eSign has started, and also the FinalFile message that is sent when uploading the edited file to PubNub. We use these to allow clients and power users to set up the iframe.

Let’s start by adding the URL of the n.eko web client, http://localhost:8080, in our video container and send it to the ESignContainer component as a prop. Next, we need to get back to the PubNub listeners to make sure that we initialize the iframe when we receive the notifications.

As mentioned before, the clients need to listen for the StartEsign message, while power users need to listen for FinalFile, so let’s add the code for setting up the iframe there.

To set up the iframe, we need the URL of the n.eko web client and also pass some query parameters that allow users to go straight to browsing. These parameters are a display name and a password.

Display user name can be whatever you want, it’s used to identify users within the co-browsing session. You can use the name your application gives to your users. Passwords, on the other hand, are defined at n.eko level. By default, admin is the password that grants elevated privileges to users and neko is for regular ones.

// the video call component
export const MeetingContainer = function MeetingContainer() {
  ...
  // useState hook for storing URL of n.eko
  const nekoUrl = "http://localhost:8080"
  const [nekoSrc, setNekoSrc] = useState()
  ...
  // useEffect hook where we define PubNub listeners
  useEffect(() => {
    ...
    // different subscriptions types for users and power users
    if (user.role === 'client') {
      listenerParams = {
        ...
        // we listen for messages
        message(s) {
          if (s.message.title === "StartEsign") {
            const src = `${nekoUrl}?usr=${encodeURIComponent( username // or whatever other value your application uses )}&pwd=neko`
            setNekoSrc(src)
          }
        }
      }
    } else {
      listenerParams = {
        file(event) {
          setFile(event.file.url)
          if (event.message === "FinalFile") {
            const src = `${nekoUrl}?usr=${encodeURIComponent( username // or whatever other value your application uses )}&pwd=admin`
            setNekoSrc(src)
          }
        }
      }
    }
    ...
  }, [pubnub, channel])
  ...
  // we pass nekoSrc as prop to ESignContainer
  return (
    ...
    <ESignContainer
      ...
      nekoSrc={nekoSrc} />
    ...
  )

Now, in ESignContainer, we just need to add the HTML iframe element when setAndShareStart value is true and set its src attribute to the prop we just created.

export const ESignContainer = function ESignContainer({
  ...
  nekoSrc
}: Props) {
  ...
  return (
    ...
    {shareAndStart && (
      <iframe 
        title="neko"
        src={nekoSrc}
        width="100%"
        height="100%"
    )}
    ...
  )
}

After that it’s just a matter of manually navigating to the ESignPageContainer page we created before. You can also build a custom web client for n.eko that supports specifying a target page as query params. If you’re interested in the latter you can check out the code of a neko-client that you can work upon.

And there you have it!

With co-browsing and PDF editing enabled in a WebRTC application, all parties can remotely sign a legal document, close a loan, perform an eMortgage, and other activities that normally require in-person witnessing, notary, or visual proof of identity.

If you’re interested in adding co-browsing and PDF editing capabilities to your WebRTC application, our team has plenty of experience! Contact us to learn how our services can adapt to your business needs. Let’s make it live!

Related post: Live eSign Saves a Trip to the Bank

View full post