7023fcd14a
feature: removed example To focus on webfuse library, daemon, provider and web based example are moved into separate repositories: daemon: https://github.com/falk-werner/webfused provider: https://github.com/falk-werner/webfuse-provider example: https://github.com/falk-werner/webfuse-example |
||
---|---|---|
.github/ISSUE_TEMPLATE | ||
.settings | ||
build | ||
cmake | ||
doc | ||
include | ||
lib/webfuse | ||
test | ||
.cproject | ||
.gitignore | ||
.project | ||
.travis.yml | ||
AUTHORS | ||
CMakeLists.txt | ||
codecov.yml | ||
COPYING.LESSER | ||
Makefile | ||
README.md | ||
VERSION |
webfuse
webfuse combines libwebsockets and libfuse. It allows ot attach a remote filesystem via websockets.
Contents
Motivation
Many embedded devices, such as smart home or IoT devices are very limited regarding to their (non-volatile) memory resources. Such devices are typically comprised of an embedded linux and a small web server, providing an interface for maintenance purposes.
Some use cases, such as firmware update, require to transfer (larger) files to the device. The firmware file is often stored multiple times on the device:
- cached by the web server, e.g. lighttpd
- copied to locally, e.g. /tmp
- uncompressed, also to /tmp
Techniques like SquashFS help to avoid the third step, since the upgrade file can be mounted directly. RAUC shows the use of SquashFS within an update facility.
However at least one (unecessary) copy of the upload file is needed on the device.
To avoid Steps 1 and 2, it would be great to keep the update file entirely in web server, just like NFS or WebDAV. Unfortunately, NFS is not based on any protocol, natively usable by a web application. WebDAV is based on HTTP, but it needs a server providing the update file.
webfuse solves this problem by using the WebSocket protocol. The emdedded device runs a service, known as webfuse adapter, awaiting incoming connections, e.g. from a web browser. The browser acts as a file system provider, providing the update file to the device.
Fellow Repositories
- webfuse-example: Example of webfuse
- webfused: Reference implementation of webfuse daemon
- webfuse-provider: Reference implementation of webfuse provider
Concept
With webfuse it is possible to implement remote filesystems based on websockets. A reference implementation of such a daemon is provided within the examples. The picture above describes the workflow:
-
The websocket filesystem daemon (webfuse daemon) waits for incoming connections.
-
A remote filesystem provider connects to webfuse daemon via websocket protocol and adds one or more filesystems.
Note: the examples include such a provider implemented in HTML and JavaScript. -
Whenever the user makes filesystem requests, such as ls, the request is redirected via webfuse daemon to the connected filesystem provider
Currently all requests are initiated by webfuse daemon and responded by filesystem provider. This may change in future, e.g. when authentication is supported.
Filesystem represenation
To handle multiple filesystems, that are registered by one or more providers, webfuse daemon maintains a directory structure as shown above.
-
mount_point is the entry point of the drectory structure
-
fwupdate is a name defined by the provider when filesystem was registered
Note: the picture above shows two providers, where both registered a filesystem named "fwupdate" -
<uuid> is the filesystem id choosen by webfuse daemon to distinguish different filesystems
-
default is a symbolic link maintained by webfuse daemon to identify the default filesystem
This directoy structure allows to handle multiple filesystems registered by multiple providers. It can be used as a kind of service registry, where each filesystem represents a service. The named subdirectores distinguish differend service types. The symbolic link default can be used to identify the default service and the listing of a named subdirectory can be used to list available services of a particular type.
Similar Projects
Davfs2
davfs2 is a Linux file system driver that allows to mount a WebDAV resource. WebDAV is an extension to HTTP/1.1 that allows remote collaborative authoring of Web resources.
Unlike webfuse, davfs2 mounts a remote filesystem locally, that is provided by a WebDAV server. In contrast, webfuse starts a server awaiting client connections to attach the remote file system.
API
Requests, responses and notifications
There are three types of messages, used for communication between webfuse daemon and filesystem provider. All message types are encoded in JSON and strongly inspired by JSON-RPC.
Request
A request is used by a sender to invoke a method on the receiver. The sender awaits a response from the receiver. Since requests and responses can be sendet or answered in any order, an id is provided in each request to identify it.
{
"method": <method_name>,
"params": <params>,
"id" : <id>
}
Item | Data type | Description |
---|---|---|
method_name | string | name of the method to invoke |
params | array | method specific parameters |
id | integer | id, which is repeated in response |
Response
A response is used to answer a prior request. There are two kinds of responses:
Successful Results
{
"result": <result>,
"id": <id>
}
Item | Data type | Description |
---|---|---|
result | any | request specific result |
id | integer | id, same as request |
Error notifications
{
"error": {
"code": <code>
},
"id": <id>
}
Item | Data type | Description |
---|---|---|
code | integer | error code |
id | integer | id, same as request |
Error codes
Symbolic name | Code | Description |
---|---|---|
GOOD | 0 | no error |
BAD | 1 | generic error |
BAD_NOTIMPLEMENTED | 2 | method not implemented |
BAD_TIMEOUT | 3 | timeout occured |
BAD_BUSY | 4 | resource busy |
BAD_FORMAT | 5 | invalid formt |
BAD_NOENTRY | 101 | invalid entry |
BAD_ACCESS_DENIED | 102 | access not allowed |
Notification
Notfications are used to inform a receiver about something. Unlike requests, notifications are not answered. Therefore, an id is not supplied.
{
"method": <method_name>,
"params": <params>
}
Item | Data type | Description |
---|---|---|
method_name | string | name of the method to invoke |
params | array | method specific parameters |
Requests (Adapter -> Provider)
lookup
Retrieve information about a filesystem entry by name.
webfuse daemon: {"method": "lookup", "params": [<filesystem>, <parent>, <name>], "id": <id>}
fs provider: {"result": {
"inode": <inode>,
"mode" : <mode>,
"type" : <type>,
"size" : <size>,
"atime": <atime>,
"mtime": <mtime>,
"ctime": <ctime>
}, "id": <id>}
Item | Data type | Description |
---|---|---|
filesystem | string | name of the filesystem |
parent | integer | inode of parent directory (1 = root) |
name | string | name of the filesystem object to look up |
inode | integer | inode of the filesystem object |
mode | integer | unix file mode |
type | "file" or "dir" | type of filesystem object |
size | integer | required for files; file size in bytes |
atime | integer | optional; unix time of last access |
mtime | integer | optional; unix time of last modification |
ctime | intefer | optional; unix time of last metadata change |
getattr
Get file attributes.
webfuse daemon: {"method": "getattr", "params": [<filesystem>, <inode>], "id": <id>}
fs provider: {"result": {
"mode" : <mode>,
"type" : <type>,
"size" : <size>,
"atime": <atime>,
"mtime": <mtime>,
"ctime": <ctime>
}, "id": <id>}
Item | Data type | Description |
---|---|---|
filesystem | string | name of the filesystem |
inode | integer | inode of the filesystem object |
mode | integer | unix file mode |
type | "file" or "dir" | type of filesystem object |
size | integer | required for files; file size in bytes |
atime | integer | optional; unix time of last access |
mtime | integer | optional; unix time of last modification |
ctime | intefer | optional; unix time of last metadata change |
readdir
Read directory contents.
Result is an array of name-inode pairs for each entry. The generic entries
"." and ".." should also be provided.
webfuse daemon: {"method": "readdir", "params": [<filesystem>, <dir_inode>], "id": <id>}
fs provider: {"result": [
{"name": <name>, "inode": <inode>},
...
], "id": <id>}
Item | Data type | Description |
---|---|---|
filesystem | string | name of the filesystem |
dir_inode | integer | inode of the directory to read |
name | integer | name of the entry |
inode | integer | inode of the entry |
open
Open a file.
webfuse daemon: {"method": "readdir", "params": [<filesystem>, <inode>, <flags>], "id": <id>}
fs provider: {"result": {"handle": <handle>}, "id": <id>}
Item | Data type | Description |
---|---|---|
filesystem | string | name of the filesystem |
inode | integer | inode of the file |
flags | integer | access mode flags (see below) |
handle | integer | handle of the file |
Flags
Symbolic name | Code | Description |
---|---|---|
O_ACCMODE | 0x003 | access mode mask |
O_RDONLY | 0x000 | open for reading only |
O_WRONLY | 0x001 | open for writing only |
O_RDWR | 0x002 | open for reading an writing |
O_CREAT | 0x040 | create (a new) file |
O_EXCL | 0x080 | open file exclusivly |
O_TRUNC | 0x200 | open file to truncate |
O_APPEND | 0x400 | open file to append |
close
Informs filesystem provider, that a file is closed.
Since close
is a notification, it cannot fail.
webfuse daemon: {"method": "close", "params": [<filesystem>, <inode>, <handle>, <flags>], "id": <id>}
Item | Data type | Description |
---|---|---|
filesystem | string | name of the filesystem |
inode | integer | inode of the file |
handle | integer | handle of the file |
flags | integer | access mode flags (see open) |
read
Read from an open file.
webfuse daemon: {"method": "close", "params": [<filesystem>, <inode>, <handle>, <offset>, <length>], "id": <id>}
fs provider: {"result": {
"data": <data>,
"format": <format>,
"count": <count>
}, "id": <id>}
Item | Data type | Description |
---|---|---|
filesystem | string | name of the filesystem |
inode | integer | inode of the file |
handle | integer | handle of the file |
offset | integer | Offet to start read operation |
length | integer | Max. number of bytes to read |
data | integer | handle of the file |
format | string | Encoding of data (see below) |
count | integer | Actual number of bytes read |
Format
Format | Description |
---|---|
"identiy" | Use data as is; note that JSON strings are UTF-8 encoded |
"base64" | data is base64 encoded |
Requests (Provider -> Adapter)
add_filesystem
Adds a filesystem.
fs provider: {"method": "add_filesytem", "params": [<name>], "id": <id>}
webfuse daemon: {"result": {"id": <name>}, "id": <id>}
Item | Data type | Description |
---|---|---|
name | string | name and id of filesystem |
authtenticate
Authenticate the provider.
If authentication is enabled, a provider must be authenticated by the adapter before filesystems can be added.
fs provider: {"method": "authenticate", "params": [<type>, <credentials>], "id": <id>}
webfuse daemon: {"result": {}, "id": <id>}
Item | Data type | Description |
---|---|---|
type | string | authentication type (see below) |
credentials | object | credentials to authenticate |
authentication types
- username: authenticate via username and password
{"username": <username>, "password": <password>}
Authentication
By default, webfuse daemon will redirect each filesystem call to the first connected provider without any authentication. This might be good for testing purposes or when an external authentication mechanism is used. In some use cases, explicit authentication is needed. Therefore, authentication can be enabled within webfuse daemon.
When authentication is enabled, filesystem calls are only redirected to a connected provider, after authenticate
has succeeded.
Enable authentication
Authentication is enabled, if one or more authenticators are registered via wf_server_config
.
static bool authenticate(struct wf_credentials * creds, void * user_data)
{
char const * username = wf_credentials_get(creds, "username");
char const * password = wf_credentials_get(creds, "password");
return ((NULL != username) && (0 == strcmp(username, "bob")) &&
(NULL != password) && (0 == strcmp(password, "???")));
}
wf_server_config * config = wf_server_config_create();
wf_server_config_add_authenticator(config, "username", &authenticate, NULL);
wf_server * server = wf_server_create(config);
//...
Authenticator types and credentidals
Each authenticator is identified by a user defined string, called type
. The type is provided by the authenticate
request, so you can define different authenticators for different authentication types, e.g. username, certificate, token.
Actually, only one type is used: username
It is strongly recommended to prefix custom authenticator types with an underscore (_
) to avoid name clashes.
The wf_credentials
struct represents a map to access credentials as key-value pairs, where both, key and value, are of type string.
username
The authenticator type username is used to authenticate via username and password. Valid credentials should contain two keys.
- username refers to the name of the user
- password refers to the password of the user
Note that no further encryption is done, so this authenticator type should not be used over unencrypted websocket connections.
Build
To install dependencies, see below.
cd webfuse
mkdir .build
cd .build
cmake ..
make
Build options
By default, unit tests are enabled. You can disable them using the following cmake options:
- WITHOUT_TESTS: disable tests
cmake -DWITHOUT_TESTS=ON ..
Dependencies
- libfuse3
- libwebsockets
- Jansson
- GoogleTest (optional)
Installation from source
libfuse
To install libfuse, meson is needed. Please refer to meson quick guide for setup instructions.
wget -O fuse-3.8.0.tar.gz https://github.com/libfuse/libfuse/archive/fuse-3.8.0.tar.gz
tar -xf fuse-3.8.0.tar.gz
cd libfuse-fuse-3.8.0
mkdir .build
cd .build
meson ..
ninja
sudo ninja install
libwebsockets
wget -O libwebsockets-3.2.0.tar.gz https://github.com/warmcat/libwebsockets/archive/v3.2.0.tar.gz
tar -xf libwebsockets-3.2.0.tar.gz
cd libwebsockets-3.2.0
mkdir .build
cd .build
cmake ..
make
sudo make install
Jansson
wget -O libjansson-2.12.tar.gz https://github.com/akheron/jansson/archive/v2.12.tar.gz
tar -xf libjansson-2.12.tar.gz
cd jansson-2.12
mkdir .build
cd .build
cmake ..
make
sudo make install
GoogleTest
Installation of GoogleTest is optional webfuse library, but required to compile tests.
wget -O gtest-1.10.0.tar.gz https://github.com/google/googletest/archive/release-1.10.0.tar.gz
tar -xf gtest-1.10.0.tar.gz
cd googletest-release-1.10.0
mkdir .build
cd .build
cmake ..
make
sudo make install