Update Sourcify IPFS docs; change Sourcify integration to use public gateway by default

docs/ipfs.md

@@ -0,0 +1,65 @@
+# IPFS
+
+This doc describes various topics regarding our use of IPFS (currently only used for Sourcify integration).
+
+## Default public gateway by default
+
+By default every IPFS resource is linked to the default public IPFS gateway at https://ipfs.io.
+
+This is done so even users without a local node can use IPFS integration by default.
+
+However this brings us some privacy concerns, since queries are made against the public gateway.
+
+We recommend the use of [IPFS Companion](https://docs.ipfs.io/install/ipfs-companion/), a popular browser extension that intercepts gateway calls and redirect them to your local gateway.
+
+This also solves the problem of users not having a local gateway at http://localhost:8080 (the default bind address of `ipfs` daemon, but also a very common bind address for other software leading to collisions), but instead having a private gateway inside the local network. The IPFS Companion allows you to customize which gateway you want it to redirect requests to.
+
+This way we can provide an out-of-box usable integration, but still allowing users to address privacy concerns.
+
+## Slow IPNS resolution
+
+By default IPNS resolution _may_ be **extremelly** slow. There is an experimental IPFS feature called IPNS pubsub that speeds it up significantly (https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#ipns-pubsub), but for it to work, it needs to be enabled in **both** resolver and publisher nodes.
+
+We asked Sourcify to enable it in their publisher node, so you just have to enable it in your local node to have an usable performance.
+
+On IPFS Companion, make sure **both** `Enable PubSub` and `Enable IPNS over PubSub` options are enabled.
+
+![IPFS pubsub](./ipns-pubsub.png)
+
+## Pinning Sourcify locally
+
+If you have an IPFS node, you can speedup things significantly and also help to spread their data among the network by pinning their entire repository.
+
+However, due to their repository characteristics (> 5 GB of thousands of small files and many nested directories), there are a few gotchas.
+
+> Please note this is based on Otterscan author's experience, it would be welcome to have more datapoints.
+
+### Use `badgerds`
+
+By default ipfs uses the `flatfs` datastore, which stores objects in simple files.
+
+I simply couldn't finish pinning the entire repo using the default settings, after 6 hours it was able to get only ~10% of the total repo, the pinning process itself slows down the computer a lot.
+
+`badgerds` is an alternative ipfs node repository format, still labeled as experimental, but marked as a _"to be turned into the default"_ repository format in future. It uses a key-value database internally.
+
+You can create a repository from scratch by using `ipfs init -p badgerds`, or convert an existing repository using [ipfs-ds-convert](https://github.com/ipfs/ipfs-ds-convert).
+
+> Use it at your own risk!
+
+My experiments pinning the entire repo on `badgerds` gave different timings, varying from 1 to 2 hours in a standard MacBook Pro laptop, totally affordable for home users, with no perceived system degradation.
+
+```
+$ time ipfs pin add --progress /ipns/k51qzi5uqu5dll0ocge71eudqnrgnogmbr37gsgl12uubsinphjoknl6bbi41p
+pinned QmVn7fcwo4Eai19hRX6dG9jAV8piHyxcrPTuobyZjEKhMW recursively
+ipfs pin add --progress   4.47s user 1.85s system 0% cpu 2:22:41.00 total
+```
+
+### IPFS root changes hourly
+
+The Sourcify root IPNS name is `/ipns/k51qzi5uqu5dll0ocge71eudqnrgnogmbr37gsgl12uubsinphjoknl6bbi41p`.
+
+Pinning it resolves it to the current IPFS root hash. However, any change to their contents changes the root hash (e.g., new contracts are verified).
+
+Sourcify current runs a cron job the updates the IPNS name hourly. It means that your pin will eventually become stale.
+
+But we think it is still worth pinning the repo because data always gets added to their repo, so by pinning current data you are speeding up your own queries to existing contracts and contributing to spread their data over the network so other people don't have to rely 100% on their node availability.

docs/sourcify.md

@@ -2,32 +2,32 @@
 
 We get the contract source code and metadata from [Sourcify](https://sourcify.dev/).
 
-There are multiple ways to consume their data we support, each one with pros and cons:
+## Integration modes
 
-## IPNS/IPFS
+There are multiple ways to consume their data that we support, each one with pros and cons:
+
+### IPFS
 
 This is the default integration method, we resolve the public Sourcify IPNS to get the latest known IPFS root hash of their repository.
 
 The downside is that recently verified contracts may not have yet been added to the root hash and republished into IPNS.
 
-It assumes a local IPFS gateway at localhost:8080 to avoid leaking your queries to public gateways.
+It uses the public gateway at https://ipfs.io by default.
 
-> This option is actually not working, but it is provided for completeness, follow https://github.com/ethereum/sourcify/issues/495
+Please see our [ipfs integration docs](./ipfs.md) for more info about how we handle all IPFS integrations and privacy concerns.
 
-## Direct HTTP connection to Sourcify's repository
+### Direct HTTP connection to Sourcify's repository
 
 Standard HTTP connection to their repo at https://repo.sourcify.dev/
 
 Fast access to fresh verified data. On the other hand it is less private and centralized.
 
-## Local snapshot
+### Local snapshot **(deprecated; soon to be removed)**
 
 As a midterm solution, we are making available a snapshot docker image of their repository, containing only mainnet full verified contracts.
 
 This would allow you to play with existing contracts up to the snapshot date/time locally, not depending on their service or IPFS connectivity availability.
 
-> It is very likely this run mode will be deprecated in future.
-
 The Sourcify snapshot is provided as a nginx image at: https://hub.docker.com/repository/docker/otterscan/sourcify-snapshot
 
 You can run it with:

src/url.ts

@@ -27,13 +27,13 @@ export enum SourcifySource {
 
 const sourcifyIPNS =
   "k51qzi5uqu5dll0ocge71eudqnrgnogmbr37gsgl12uubsinphjoknl6bbi41p";
-const ipfsGatewayPrefix = `http://localhost:8080/ipns/${sourcifyIPNS}`;
+const defaultIpfsGatewayPrefix = `https://ipfs.io/ipns/${sourcifyIPNS}`;
 const sourcifyHttpRepoPrefix = `https://repo.sourcify.dev`;
 const snapshotPrefix = "http://localhost:3006";
 
 const resolveSourcifySource = (source: SourcifySource) => {
   if (source === SourcifySource.IPFS_IPNS) {
-    return ipfsGatewayPrefix;
+    return defaultIpfsGatewayPrefix;
   }
   if (source === SourcifySource.CENTRAL_SERVER) {
     return sourcifyHttpRepoPrefix;