diff --git a/CMakeLists.txt b/CMakeLists.txt index fc41227..966db70 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1,6 +1,7 @@ cmake_minimum_required(VERSION 3.10) project(webfuse VERSION 2.0.0) +option(WITHOUT_PROVIDER "Disabled build of webfuse provider" OFF) option(WITHOUT_TEST "Disables unit and integration tests" OFF) option(WITHOUT_CLANG_TIDY "Disables clang tidy" OFF) @@ -43,17 +44,16 @@ set_property( endif() -add_executable(webfuse - src/main.cpp) - +add_executable(webfuse src/main.cpp) target_link_libraries(webfuse PRIVATE webfuse_static) +install(TARGETS webfuse DESTINATION bin) + if(NOT(WITHOUT_PROVIDER)) -add_executable(webfuse_provider - src/provider_main.cpp) - +add_executable(webfuse_provider src/provider_main.cpp) target_link_libraries(webfuse_provider PRIVATE webfuse_static) +install(TARGETS webfuse_provider DESTINATION bin) endif() diff --git a/README.md b/README.md index 2d68498..6b7dde4 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,72 @@ [![build](https://github.com/falk-werner/webfuse/actions/workflows/build.yml/badge.svg)](https://github.com/falk-werner/webfuse/actions/workflows/build.yml) -# webfuse2 +# webfuse -Reimplementation of webfuse. +webfuse combines libwebsockets and libfuse. It allows to attach a remote filesystem via websockets. -## Build +## Motivation -```` -cmake -B build -cmake --build build -```` \ No newline at end of file +Many embedded devices, such as smart home or [IoT](https://en.wikipedia.org/wiki/Internet_of_things) devices are very limited regarding to their (non-volatile) memory resources. Such devices are typically comprised of an embedded linux and a small web server, providing an interface for maintenance purposes. + +Some use cases, such as firmware update, require to transfer (larger) files to the device. The firmware file is often stored multiple times on the device: + +1. cached by the web server, e.g. [lighttpd](https://redmine.lighttpd.net/boards/2/topics/3451) +2. copied to locally, e.g. /tmp +3. uncompressed, also to /tmp + +Techniques like [SquashFS](https://en.wikipedia.org/wiki/SquashFS) help to avoid the third step, since the upgrade file can be mounted directly. [RAUC](https://rauc.io/) shows the use of SquashFS within an update facility. +However at least one (unecessary) copy of the upload file is needed on the device. + +To avoid Steps 1 and 2, it would be great to keep the update file entirely in web server, just like [NFS](https://en.wikipedia.org/wiki/Network_File_System) or [WebDAV](https://wiki.archlinux.org/index.php/WebDAV). Unfortunately, NFS is not based on any protocol, natively usable by a web application. WebDAV is based on HTTP, but it needs a server providing the update file. + +webfuse solves this problem by using the [WebSocket](https://en.wikipedia.org/wiki/WebSocket) protocol. The emdedded device runs a service, known as webfuse adapter, awaiting incoming connections, e.g. from a web browser. The browser acts as a file system provider, providing the update file to the device. + +## Concept + +![concept](doc/concept.png) + +With webfuse it is possible to implement remote filesystems based on websockets. +Therefore, webfuse defined two roles participating in a webfuse connection: + +- webfuse service +- webfuse provider + +### Webfuse Service + +A `webfuse service` is both, +- a [websocket](https://en.wikipedia.org/wiki/WebSocket) service providing the `webfuse` protocol +- a [fuse](https://github.com/libfuse/libfuse) filesystem attached to a local mountpoint + +The `webfuse service` awaits incoming connections from a `webfuse provider`. Once connected, it communicates all the filesystem requests originated by the `libfuse` to the connected `webfuse provider` using the `websocket`-based `webfuse protocol`. + +By doing so, `webfuse` allows to inject a filesystem to a remote device. + +### Webfuse Provider + +A `webfuse provider` provides a filesystem to a remote device using the `websocket`-based `webfuse protocol`. Therefore, a `webfuse provider` implements a `websocket` client. + + +## Similar Projects + +### Davfs2 + +[davfs2](http://savannah.nongnu.org/projects/davfs2) is a Linux file system driver that allows to mount a [WebDAV](https://wiki.archlinux.org/index.php/WebDAV) resource. WebDAV is an extension to HTTP/1.1 that allows remote collaborative authoring of Web resources. + +Unlike webfuse, davfs2 mounts a remote filesystem locally, that is provided by a WebDAV server. In contrast, webfuse starts a server awaiting client connections to attach the remote file system. + +## Further Documentation + +- [Build instructions](doc/build.md) +- [Webfuse Protocol](doc/protocol.md) + +## webfuse legacy + +`webfuse2` is a complete re-implementation of the original idea behind `webfuse`. In contrast to the original `webfuse` implementation, `webfuse2` provides also write access to the filesystem and allows to use all standard options of a fuse filesystem. + +But `webfuse2` marks also some breaking changes: + +- `webfuse2` uses a new, binary protocol which is not compatible to the JSON-based protocol of legacy `webfuse` +- `webfuse` does not provide an API nor a library to program against + _(if you are interested in an API or a library for webfuse2 feel free to create an issue)_ + +When you are interested in the original `webfuse` implementation take a look at it [branch](https://github.com/falk-werner/webfuse/tree/master). diff --git a/doc/README.md b/doc/README.md new file mode 100644 index 0000000..f193966 --- /dev/null +++ b/doc/README.md @@ -0,0 +1,4 @@ +# webfuse developer documentation + +- [Build instructions](build.md) +- [Webfuse2 protocol](protocol.md) diff --git a/doc/build.md b/doc/build.md new file mode 100644 index 0000000..2f40343 --- /dev/null +++ b/doc/build.md @@ -0,0 +1,35 @@ +# webfuse build instructions + +## Build + +```` +cmake -B build +cmake --build build +sudo cmake --install build +```` + +## Build options + +| Options | Default | Description | +| ------------------ | -------- | ----------- | +| WITHOUT_PROVIDER | OFF | Disables build of webfuse provider | +| WITHOUT_TEST | OFF | Disables build of unit and integration tests | +| WITHOUT_CLANG_TIDY | OFF | Disables running clang tidy on build | + +## Dependencies + +- [libfuse](https://github.com/libfuse/libfuse) +- [libwebsockets](https://libwebsockets.org/) + +### Optional dependencies + +- [googletest](https://github.com/google/googletest) +- [valgrind](https://valgrind.org/) +- [clang-tidy](https://clang.llvm.org/extra/clang-tidy/) + +## Additional cmake targets + +| Target | Description | +| -------- | ----------- | +| test | runs unit and integration tests | +| memcheck | runs unit test with valgrind/memcheck | diff --git a/doc/concept.png b/doc/concept.png new file mode 100644 index 0000000..bf5b875 Binary files /dev/null and b/doc/concept.png differ diff --git a/doc/concept.uml b/doc/concept.uml new file mode 100644 index 0000000..e4dcb20 --- /dev/null +++ b/doc/concept.uml @@ -0,0 +1,26 @@ +@startuml +participant "webfuse provider" as provider +participant "webfuse service" as service +actor "user" as user + +group startup +service -> service : fuse_mount +service -> service : start websocket server +end +... + +group connect +provider -> service : connect +end +... + + +group directory listing +user -> service : ls +service -> provider : readdir request +provider --> service : readdir response +service --> user : [., ..] +end +... + +@enduml \ No newline at end of file diff --git a/doc/protocol.md b/doc/protocol.md index 9993862..b6cda83 100644 --- a/doc/protocol.md +++ b/doc/protocol.md @@ -2,34 +2,38 @@ ## Scope -This document describes the webfuse 2 communication protocol. The protocol is used to transfer messages between a `webfuse2 service` and a `webfuse2 provider`. In contrast to `legacy webfuse`, which is based on `JSON RPC` the `webfuse2 protocol` is a binary protocol. +This document describes the webfuse 2 communication protocol. The protocol is used to transfer messages between a `webfuse service` and a `webfuse provider`. In contrast to `legacy webfuse`, which is based on `JSON RPC` the `webfuse2 protocol` is a binary protocol. ## Definitions -### Webfuse2 Service +### Webfuse Service -A `webfuse2 service` is both, -- a [websocket](https://en.wikipedia.org/wiki/WebSocket) service providing the `webfuse2` protocol +A `webfuse service` is both, +- a [websocket](https://en.wikipedia.org/wiki/WebSocket) service providing the `webfuse` protocol - a [fuse](https://github.com/libfuse/libfuse) filesystem attached to a local mountpoint -The `webfuse2 service` awaits incoming connections from a `webfuse2 provider`. Once connected, it communicates all the filesystem requests originated by the `libfuse` to the connected `webfuse2 provider` using the `websocket`-based `webfuse2 protocol`. +The `webfuse service` awaits incoming connections from a `webfuse provider`. Once connected, it communicates all the filesystem requests originated by the `libfuse` to the connected `webfuse provider` using the `websocket`-based `webfuse protocol`. -By doing so, `webfuse2` allows to inject a filesystem to a remote device. +By doing so, `webfuse` allows to inject a filesystem to a remote device. -### Webfuse2 Provider +### Webfuse Provider -A `webfuse2 provider` provides a filesystem to a remote device using the `websocket`-based `webfuse2 protocol`. Therefore, a `webfuse2 provider` implements a `websocket` client. +A `webfuse provider` provides a filesystem to a remote device using the `websocket`-based `webfuse protocol`. Therefore, a `webfuse provider` implements a `websocket` client. + +## Websocket protocol name + +The webfuse2 protocol uses the following websocket protocol name: `webfuse2`. ## Message exchange Once connected, the `webfuse2 protocol` implements a strict request-response scheme, where -- all requests are send by the `webfuse2 service`, +- all requests are send by the `webfuse service`, - all requests require a response and -- all responses are send by the `webfuse2 provider` +- all responses are send by the `webfuse provider` -Note that this communication is reversed the typical client-server-communication scheme. In `webfuse2` the `webfuse2 serive` (server) sends all the requests and the `webfuse provider` (client) sends the responses. +Note that this communication is reversed the typical client-server-communication scheme. In `webfuse` the `webfuse service` (server) sends all the requests and the `webfuse provider` (client) sends the responses. -For message transfer, the [websocket](https://en.wikipedia.org/wiki/WebSocket) protocol is used. All messages are in binary form, plain text messages are never used by the `webfuse2 protocol`. +For message transfer, the [websocket](https://en.wikipedia.org/wiki/WebSocket) protocol is used. All messages are in binary form, plain text messages are never used by the `webfuse protocol`. ## Endianness @@ -259,27 +263,27 @@ _Note that the following numbers are in `octal` notation._ | type | u8 | Type of the message | | payload | u8[] | Payload according to the message type | -The `id` is just a number without any meaning for the `webfuse2 provider`. It is set by the `webfuse2 service` of a request and is copied by the `webfuse2 provider` to the response. A `webfuse2 service` implementation might choose to keep track on pending requests using the `id`. +The `id` is just a number without any meaning for the `webfuse provider`. It is set by the `webfuse service` of a request and is copied by the `webfuse provider` to the response. A `webfuse service` implementation might choose to keep track on pending requests using the `id`. ### Erroneous Responses -Most responses contain a `result` encoding the status of the operation. While successful responses may contain additional data, erroneous responses must not be decoded by a `webfuse2 service` implementation beyond the `result` value. +Most responses contain a `result` encoding the status of the operation. While successful responses may contain additional data, erroneous responses must not be decoded by a `webfuse service` implementation beyond the `result` value. ### Unknown requests -There are two reserved message types in `webfuse2`: +There are two reserved message types: - **0x00:** Unknown request - **0x80:** Unknown response -A `webfuse2 service` may send a request of type `unknown request` for conformance testing reasons. +A `webfuse service` may send a request of type `unknown request` for conformance testing reasons. -Since each request requires a response, a `webfuse2 provider` must respond to any unknown requests with a message of `unknown response` type. This allows to add new request types in future. +Since each request requires a response, a `webfuse provider` must respond to any unknown requests with a message of `unknown response` type. This allows to add new request types in future. ### Accept additional data in requests -Both, a `webfuse2 provider` and a `webfuse service` must accept messages that contain more data than specified. This allows to add optional fields to existing requests and / or responses in future. +Both, a `webfuse provider` and a `webfuse service` must accept messages that contain more data than specified. This allows to add optional fields to existing requests and / or responses in future. -_Note there are no optional fields in the current revision of the `webfuse2 protocol`. +_Note there are no optional fields in the current revision of the `webfuse2 protocol` yet._ ### Message Types @@ -312,7 +316,7 @@ _Note that the following numbers are in `hexadecimal` notation._ ## Methods -Since `webfuse2` aims to communicate the `libfuse API` over a `websocket` connection, `webfuse2` methods are tightly connected to [fuse operations](https://libfuse.github.io/doxygen/structfuse__operations.html) which itself have a tight connection to `posix filesystem operations`. Therefore, additional information about most `webfuse2` operations can be found in the [fuse operations documentation](https://libfuse.github.io/doxygen/structfuse__operations.html) and / or the [man pages](https://man7.org/index.html). +Since `webfuse` aims to communicate the `libfuse API` over a `websocket` connection, `webfuse` methods are tightly connected to [fuse operations](https://libfuse.github.io/doxygen/structfuse__operations.html) which itself have a tight connection to `posix filesystem operations`. Therefore, additional information about most `webfuse` operations can be found in the [fuse operations documentation](https://libfuse.github.io/doxygen/structfuse__operations.html) and / or the [man pages](https://man7.org/index.html). ### access