How Neighbor Works?

Neighbor protocol is not really a new “protocol” itself, is a group of endpoints that allow clients communicate between them, and between communities allowing to search torrents, announce news torrents, and share other clients and communities.

The main idea of the Neighbor protocol are the “simplicity” in the protocol itself, allowing any developer create communities or clients in any language, and implementing the protocol in any actual program in the easiest possible way.


What is a "Community"?

One “Community” is a server that implement some of this protocol endpoints, allow clients to announce new torrents or search contents in the community using multiple filters.

To allow create communities fast and in easy way, you have a base community server in PHP available in this Github Repository

This server is done using Phalcon to speed up the maximum possible request per second, the Source Code is under MIT license so you can do what you like with the code.

You can see and test this with my personal community I do for my family with legal torrents extracted from epublibre.com: https://community.benmaster.es/

Note: Please if you found no legal torrents inside the community, send an email to webmaster @ benmaster.es to remove from the server, the content is updated automatically.


Protocol Basis

All the endpoints request and responses return the data in JSON format, using UTF-8 encoding.

By default all returned responses from the endpoints need to have the "state" param, to check the response status, anyway check always the response headers.

Possible states to return in the endpoint responses:

If something go wrong, return error with 'internal server error' and 500 Header, if the error happens because the data is wrong, return error with 'malformed or wrong data' and 422 Header.


Token

To allow make search's to multiple clients and communities and don't care about if the request is reply in any order or is never reply, all search contain a "token" param, this param allow to verify if the client/community reply is for the actual search or is for other search, and prevent malicious clients insert arbitrary contents in the actual search.


Endpoints

Here is the list of available endpoints of the protocol and their structure. This can be changed because is a Work In Progress protocol


GET /library

This endpoint accept only one param "page". If the page is not set, the endpoint return only the total torrents available in the client or community:

GET /library

    {
        "state": "ok",
        "count": 2941
    }
        

If the "page" param is set, the endpoint return a list of torrents order by creation date DESC (last insert, first returned), and with the page value you can retrieve every page of the total stored in the server.


GET /library/search

This endpoint allow to search torrents using different params to filter the contents.

Param Description
search

The search query, by default use the name and description field to search on it using LIKE '%{search}%' query.

tags

Coma separated tag names, example: "music,movie,tv,action"

languages

Coma separated ISO(3) languages, example: "eng,spa"

page

The number of the page you want over the total.

limit

The results per page limit of the search. Note: some clients/communities can have a fixed limit, check always the limit field returned in the response.

mindate

Search torrent with the same or greater date than this.

maxdate

Search torrent with the same or lower date than this.

token

This is used for searchs. The value returned in the response need to be the same of the param, if dont match, can be a security issue.


GET /library/search?search=aaa&tags=a,b,c
GET /library?page=1

    {
        "state": "ok",
        "total": 2941,
        "limit": 100,
        "page": 1,
        "rows": [
            {
                "id": "9999",
                "hash": "ef851680a21f72459b7161180615153ba886b2c1",
                "name": "Lorem ipsum dolor",
                "magnet": "magnet:?xt=urn:btih:ef859680a21f...",
                "description": "Lorem ipsum dolor sit amet.",
                "tags": [
                    "ebook",
                    "novel",
                    "adventure",
                ],
                "languages": [
                    "eng"
                ],
                "date": "1509559080",
                "metadata": {
                    "image": {
                        "type": "image",
                        "value": "https://www.domain.com/images/cover.jpg"
                    },
                    "size": {
                        "type": "bytes",
                        "value": 1982910
                    }
                }
            },
            ...
        ]
    }
        

POST /library/announce

This endpoint allow to send new torrents to other clients or communities, using POST and sending in the BODY of the request the JSON with the torrents in an array.


    [
        {
            "hash": "123456A...",
            "name": "lorem ipsum",
            "description": "lorem ipsum dolor sit",
            "tags": ["one","two"],
            "languages": ["spa","end"]
        },
        ...
    ]
        

If some of the torrents are stored, the response is an array of hashes stored, the count, and the state


    {
        "state": "ok",
        "hashes": ["aef222","345fff","ed2323"],
        "count": 1
    }
        

if something go wrong the response state is different of "ok" and can contain a meessage. For example if the endpoint is disables in a community the response is like this:


    {
        "state": "disabled",
        "message": "endpoint disabled"
    }
        

GET /community

This endpoint allow to retrieve other clients communities. This feature can be disabled in the Neighbor in the two sides, you can disable the option to share your communities with others, or the feature to ask others their communities.


    {
        "state": "ok",
        "communities": ["https://www.community.com", "127.0.0.1:13375"]
    }
        

GET /client

This endpoint allow to retrieve the last clients IPs.
This feature can be disabled in the Neighbor in the two sides, you can disable the option to share your clients with others, or the feature to ask others their clients.

This feature is available in the clients and the communities, can be a security issue, but help clients to discover other clients and get network information fast.


    {
        "state": "ok",
        "clients": [
          "192.168.1.131",
          "80.80.80.80:13375"
        ]
    }
        

Storage structure

The stored data can be done in any way, but keep a base structure like this to ensure the compatibility with all programs that implement Neighbor protocol.

Field Type Info
hash String

Torrent hash identifier, complete unique.

name String

The name of the torrent or the torrent content name.

magnet String

The torrent magnet itself, because we don't store the torrents, only the magnets

description String

The torrent description, can contain HTML but is better if is not to long.

tags JSON ARRAY

JSON array of tags as "categories", this categories are open and are based on common torrent websites like "ebook", "movies", "audio", "terror", "HD", etc

languages JSON ARRAY

JSON array of languages in ISO 639-2 char codes, for example "eng", "spa", "por", etc

date Integer

The date of the torrent creation, UNIX Timestamp.

metadata JSON

To store extra torrent content this field allow store different data types using JSON, the structure is described in below.


Metada example

The metadata of the stored torrents is a simply JSON ARRAY with a list of "type" + "value" fields with different types available.

The types are not fixed, you can add the types you like, I only create some basic types for the basic examples, the implementation of the field types is in the developers hands

Type Info
image

Store image, can use base64 images or simply URL. If use base64 images use lowest quality possible to keep the metadata size low

text

Store any time of text

bytes

Store size in bytes, to convert to human readable sizes.

date

Store Unix Timestamp date, to convert in the user location date format.

html

Allow to show some HTML code. Be careful with certain HTML, like scripts, frame or iframes

iframe

HTML Iframe to allow show external content, for example website comercial banners to help the torrent pages.Be careful with the XSS Attacks

list

JSON ARRAY list like "[{"a":1,"b":2},{"c":"cccc","d":"dddd"}]" to show in loop

This is an example of metadata object:


    "metadata": {
        "source": {
            "type": "text",
            "value": "https://www.google.es"
        },
        "gender": {
            "type": "text",
            "value": "Action"
        },
        "resource_from": {
            "type": "html",
            "value": "Downloaded from www.google.es"
        },
        "size": {
            "type": "bytes",
            "value": 106164228
        }
    }