When we launched Web3.Storage in August 2021, we knew we were onto something great, and our users have reflected that back to us. By making a simple "on-ramp" to IPFS and Filecoin, web3.storage has enabled traditional web developers to tap into the decentralized ecosystem without having to run their own storage infrastructure or learn the nuances of decentralized storage protocols.

Since then, we've been hard at work building a new application platform with content addressing at its heart, with a powerful new authorization protocol powered by UCAN, or User Controlled Authorization Networks.

Today we're announcing the first component of the new Web3.Storage platform, a file upload service called w3up. w3up is a complete reimagining of the Web3.Storage API that leverages the strengths of UCAN authorization and our cloud-native Elastic IPFS stack.

Yes, UCAN!

UCAN is a central component of our new platform, and we think it unlocks some pretty amazing super powers. But what is a UCAN, and why should you care about it?

UCANs are a relatively new authorization protocol developed by Fission that allows you to build fine-grained permission and authorization systems without requiring a single "issuer" or central authority.

In a traditional authorization system, you sign up for an account from some provider, and they'll create an entry for you in their database using whatever ID fits their internal models. In some cases, you might be able to sign in with some other ID ("Sign in with Google", etc.), but that's just passing the buck to another third party who controls your identity.

With UCAN, you hold the keys, in a very literal sense. Your UCAN identity is based on a public key, the private half of which is only known to you. When you register your UCAN identity with w3up, you tell us who you are, not the other way around. There's no way for us or anyone else to impersonate your account without your private key.

That's already pretty cool, but what makes UCAN truly special is that we as the service provider are not the only ones that can issue UCAN tokens - you can too! Once you've created a w3up account and registered your UCAN identity, you can issue your own UCAN tokens and delegate some or all of your permissions to another UCAN identity. This might be another device like your phone, or it could be a user on a platform that you manage.

Using w3up-client

w3up is our first service built from the ground up on our new platform. It offers UCAN-based identity and a streamlined upload experience based on Elastic IPFS.

In this post we'll take a look at the JavaScript client library w3up-client. If you're more into the command line, check out the w3up-cli project, and stay tuned for a post that explores it in detail!

Below we'll look at a few examples of using w3up-client to register a UCAN identity and upload files. The examples are snippets of the simple-upload example project, available in the w3up-client examples repository. Check out the full source code for more details!

You can add the w3up-client package to your JavaScript or TypeScript project with npm:

npm install w3up-client

Creating a client object

The createClient function in the w3up-client package creates and returns a client object you can use to interact with the w3up service.

import { createClient } from 'w3up-client'

const client = createClient()

Registering a UCAN identity

Before you can upload, you'll need to register a UCAN identity with the w3up service.

The client's register method will generate a new UCAN identity and register it with an email address. When you call register, an email is sent to the given address with a confirmation link. The async register method will finish successfully once the link in the email is clicked.

async function tryToRegister(email) {
 try {
  console.log(`Registering email address ${email}. Please check your inbox. If you don't see the email, check your spam folder.`)
  const id = await client.identity()
  await client.register(email)
    
  console.log(`Success! Registered email ${email} with id ${id.did()}`)
  } catch (err) {
    console.error(`Registration error: ${err}`)
  }
}

Note that in a real app, you'd want to save the key for your new id somewhere. See the simple-upload example's saveSettings function for one way to do so.

Preparing your data

The w3up service accepts data in the form of Content Archives, also known as CARs.

A CAR is a collection of content addressed blocks of data, and we use them to ensure end-to-end data integrity and streamline the data processing pipeline when you upload data.

We are currently integrating the CAR chunking from w3up-cli into the client library, so soon you'll be able to use w3up-client without knowing anything about CARs, as we'll be incorporating the encoding process into the client.

For now, you can either use ipfs-car, which powers the current web3.storage client, or @ipld/unixfs, a new implementation focused on efficient encoding of large files with minimal buffering. Or you can prepare your uploads into CAR files using w3up-cli.

The simple-upload example uses ipfs-car, because the API is a little easier to use than @ipld/unixfs if you're new to IPFS and the CAR format. However, if you expect to upload large files or a directory with a large collection of small files, you will likely be better served by @ipld/car. You can also see the guide to working with CARs for more options.

The snippet below shows the prepareForUpload function from the simple-upload example, which uses the packToBlob function from ipfs-car:

/**
 * Takes the path to a file and creates a CAR with a an IPFS (UnixFS) formatted file object, wrapped in a directory so that we can preserve the filename.
 *  
 * @param {string} filename - the path to a local file to be uploaded
 * 
 * @returns {Promise<{ rootCID: string, carBytes: Uint8Array }>}
 */
async function prepareForUpload(filename) {
  // ipfs-car accepts several types of "file-like" data.
  // We're using the `ToFile` type, which expects an object with `path` and `content` fields

  // The `content` field can be a `ReadableStream<Uint8Array>`, which is conveniently what's returned by [`fs.createReadStream`](https://nodejs.org/api/fs.html#fscreatereadstreampath-options)
  const content = fs.createReadStream(filename)

  // Since our `filename` may contain the full path to a file, we use [`path.basename`](https://nodejs.org/api/path.html#pathbasenamepath-ext)
  // to get just the filename part of the path to use when packing the CAR.
  const fileBasename = path.basename(filename)

  // The `file` object below now has the fields that ipfs-car expects for its `input` argument.
  const file = { content, path: fileBasename }

  const { root, car } = await packToBlob({
    // we need to pass an iterable (in this case, an array), even when packing a single file
    input: [file] 
  })

  // packToBlob returns a [`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob), but we want
  // a `Uint8Array` to pass to the client's `upload` method.
  const buf = await car.arrayBuffer()
  const carBytes = new Uint8Array(buf)

  // root is a CID object, but we only need the string form for this example
  const rootCID = root.toString()
  return { rootCID, carBytes }
}

Uploading data

Now that you have a registered UCAN identity and know how to create a CAR with your data, it's time to upload!

Just call the upload method on the client, passing in a Uint8Array containing the encoded CAR.

Below is a function that takes a client object and the path to a file and uploads it to the service, using the prepareForUpload helper:

async function upload(client, filename) {
  const { rootCID, carBytes } = await prepareForUpload(filename)
  console.log(`Uploading ${filename}`)
  console.log(`Root CID: ${rootCID}\n`)

  try {
    await client.upload(carBytes)
    console.log('🎉 Upload complete!')

    const gatewayURL = `https://w3s.link/ipfs/${rootCID}`
    console.log(`🔗 View on an IPFS <-> HTTP gateway at ${gatewayURL}`)
    console.log('Note that it may take a few seconds before content is available.')
  } catch (err) {
    console.error(`🧨 Upload failed with an error: ${err}`)
  }
}

Next steps

Hopefully this post has piqued your interest in w3up. We're excited to put it into your hands and eager to see what you'll build.

If you want to dig deeper, make sure to check out w3up-cli, our fully-featured command line tool.

If you have any issues using w3up, we want to know all about it. Please feel free to reach out early and often with any feedback!