+
-###### 14 Apr 2020, Hendrik Niefeld
+###### 14 Apr 2020, Frederik Ring
# [Episode Three — Displaying data](/blog/displaying-data/)
-Milestone 3 has been completed. This is what we have achieved in the last ten weeks.
+
+In the middle of strange times Milestone 3 - "Displaying Data" - is done. This means we focused on how we aggregate and display the data Offen collects in a way that operators can use it to improve their services and users can understand what is being collected and what it means for their privacy.
+
+As we are now cutting official alpha releases already, we have also changed the format of our project updates. Instead of one pull request per milestone, we now have an update that spans across multiple releases. During Milestone 3 we have released the following versions:
+
+- [v0.1.0-alpha.2](https://github.com/offen/offen/releases/tag/v0.1.0-alpha.2)
+- [v0.1.0-alpha.3](https://github.com/offen/offen/releases/tag/v0.1.0-alpha.3)
+- [v0.1.0-alpha.4](https://github.com/offen/offen/releases/tag/v0.1.0-alpha.4)
+- [v0.1.0-alpha.5](https://github.com/offen/offen/releases/tag/v0.1.0-alpha.5)
+
+As always, you can download the latest release from [https://get.offen.dev](https://get.offen.dev) or pull it from Docker Hub.
---
-### Improved UX
-The Auditorium for users is now annotated with explanations of all metrics as well as analytics-specific terms. It also contains an FAQ section. In general, new metrics were added and the UX was refined for users and operators.
+## Achievements
+
+### Improved and annotated Auditorium
+
+The Auditorium is where both users and operators access and manage usage data. Operators want to gain insights in how their services are being used, users want to understand what data Offen is collecting and manage this data.
+
+The user facing Auditorium is now annotated with explanations for each metric and also explains analytics-specific terms and answers frequently asked questions. This has been implemented in [PR 339](https://github.com/offen/offen/pull/339).
+
-These updates are already *up and running in our own instance* for users. Opt in and open the Auditorium.
-[Check it out](https://analytics.offen.dev/){: target="_blank" data-button="outline"}
-### Working in teams
-If you want to improve your services with transparent and fair web analytics, it is very likely that you collaborate in groups. Therefore, in a first draft we have added basic utilities for managing user logins and accounts. These tools will be further refined during milestone 4.
+In addition to that we added new metrics and improved the overall user experience for both users and operators alike. Relevant PRs are [331](https://github.com/offen/offen/pull/331), [328](https://github.com/offen/offen/pull/328), [327](https://github.com/offen/offen/pull/327), [324](https://github.com/offen/offen/pull/324), [319](https://github.com/offen/offen/pull/319), [317](https://github.com/offen/offen/pull/317), [291](https://github.com/offen/offen/pull/291)
+
+### Account Management fundamentals
+
+Operators that are using Offen to improve their services are likely to work in teams. This is why the application needs tools for managing user logins and accounts for segmenting usage data. In a first draft we added basic versions of these features so that the absence of such capabilities doesn't further prevent the adoption of Offen. This initiative has been started in [PR 288](https://github.com/offen/offen/pull/288) and will be refined further during Milestone 4.
+
+### In-Browser setup
+
+The command line can seem daunting when you don't use it regularly. This is why we added an in-browser setup screen for Offen. Instead of performing the initial setup using the `offen setup` command from the CLI, operators who prefer to do so can now use their browser to navigate to the `/setup` URL of their installation and perform the initial setup there. We hope this helps us gaining further traction with semi-technical users that want to use privacy friendly tools. This has been implemented in [PR 299](https://github.com/offen/offen/pull/299).
-### Convenient setup
-Using the command line can be a bit intimidating. This is why we have added an in-browser setup screen for a more intuitive setup. You can now use your browser to perform the initial setup.
-### Goodbye Choo, hello Preact
-[Choo](https://choo.io/){: target="_blank"} was a great choice to build dynamic UIs very quickly. But as the Auditorium grew, we noticed a lack of mechanisms to break interfaces into components. So we migrated the application to [Preact.](https://preactjs.com/){: target="_blank"} A better way to use modern paradigms for programming interfaces.
+### Goodbye, Choo
-### Verifying download integrity
-Our binaries are now signed with a GPG key. After downloading Offen, you can now check if the binary you want to use is the one we intend to distribute.
-[Learn more](https://docs.offen.dev/running-offen/downloads-distributions/#verifying-the-binaries-signatures){: target="_blank" data-button="outline"}
+We started out with using Choo as our frontend framework and it was a great choice as it allowed us to build dynamic UIs very rapidly. But as Offen's Auditorium was growing over the last few months we noticed its lack of mechanisms for breaking interfaces into components was starting to slow us down signficantly and made simple changes cumbersome to implement.
-### Safari now fully supported
-At the end of milestone 2 we noticed problems with the way the Safari browser stores keys. We fixed this by adding a dedicated fallback mechanism. [Give it a try.](https://analytics.offen.dev/){: target="_blank"}
+This is why - before starting work on the annotated Auditorium - we migrated the application to use Preact in [PR 289](https://github.com/offen/offen/pull/289). Preact is a great choice as it allows us to use modern paradigms for programming interfaces without having to buy into a Facebook-dominated ecosystem. The library is distributed under a MIT license.
+
+### Signed binary downloads
+
+As we start to target more and more distribution channels, ensuring the integrity of what people actually download is very important for an application like Offen. With [PR 338](https://github.com/offen/offen/pull/338) we started signing our binaries with a GPG key. Downloaders can now verify that a binary they are planning to use is the one we intend to distribute:
+
+```bash
+gpg --keyserver pgp.mit.edu --recv F20D4074068C636D58B53F46FD60FBEDC90B8DA1
+gpg --verify offen-linux-amd64.asc offen-linux-amd64
+```
+
+### Safari Support
+
+Offen generates client side keys for encrypting data for each user. At the end of Milestone 2 we noticed issues where the Safari browser would not store these keys properly which means users would lose access to their user data. We fixed this in [PR 282](https://github.com/offen/offen/pull/282) by adding a fallback mechanism that Safari can use. Keys are guaranteed to be safe from third party access in both implementations.
+
+### 1-Click Deploy
+
+Self hosted software is a great fit for privacy focused software like Offen. Yet, it can seem daunting to non-technical users and make them stick to established SaaS solutions longer than needed. This is why we put a lot of effort into finding easy "1-click" options to deploy an Offen instance. In Milestone 3 we have created a 1-click solution for deploying Offen to Heroku: [https://github.com/offen/heroku](https://github.com/offen/heroku). Using free resources only, people interested in running Offen can now deploy a production ready instance to Heroku in less than 1 minute. We hope this encourages website operator to consider self-hosted software and Offen as a real option. Required changes for this were implemented in [PR 287](https://github.com/offen/offen/pull/287).
-### Ready in one minute
-Self-hosting is ideal for privacy focused software. Based on free resources, you can now deploy a production ready Offen instance to Heroku in a jiffy.
-We are currently working on further 1-click options for platforms such as [DigitalOcean](https://github.com/offen/digitalocean){: target="_blank"} and YunoHost.
-[Deploy to Heroku](https://heroku.com/deploy?template=https://github.com/offen/heroku/tree/maste){: target="_blank" data-button="outline"}
-### Happy Birthday to us
-Offen is now [one year old.](https://github.com/offen/offen/commit/3a50763bbd93c9c655fad002e94c340a623ee613){: target="_blank"} We made some progress on our way to develop a self hosted and transparent alternative to established web analytics tools.
-
-But it is not yet the time for standing ovations. There is still a lot to be done. If you like, you can help us. *Stay tuned and provide us with your feedback.* Thanks in advance!
-
-### Up next
-*Episode Four — Managing data* will feature a more fine grained account management and the collecting and implementing of first user feedback. We'll be right back.
+To offer even more options we are also looking into building a 1-click option for other platforms like YunoHost and DigitalOcean very soon.
---
-### Deep dive
-Interested in the details? Want to get your hands dirty? Take a look at our code.
-[Open GitHub repo](https://github.com/offen/offen){: target="_blank" data-button="full"}
+## Next up
+
+Milestone 4 is up next and is called "Managing Data". This means we will focus on features that allow operators and users to manage their usage data and Offen instances.
+
+This is also *where we will have a slight deviation from the original product plan*: We originally had an item called "Selective Data deletion for users", yet as we have moved to focus on self-hosting more and more, this feature does not make too much sense anymore. Instead we will allow users to delete their data tied to an instance entirely and *will make it easy to follow how deletion works and what the implications are*.
+
+### Account management
+
+In addition to the basic account management features Offen already offers, we'll work on implementing more fine grained access control mechanisms so that teams that are using Offen can easily and safely share access to an instance. In addition to that we will also revisit existing features and try to make them easier to use and more accessible.
+
+### Collecting and implementing real world user feedback
+
+We are still labeling Offen as `alpha` right now, but we think the next weeks could be the right time to slough that potentially scary label off and attract more users by going `beta`.
+
+Before we do so though, we would like to collect feedback from people that are brave enough to deploy the existing alpha version and use it for a while. Luckily there are some, so that's a good start, but in case you do know of people who might be interested in deploying and testing Offen, we'd be happy if you could spread the word. Documentation for running Offen is readily available at [https://docs.offen.dev/running-offen/](https://docs.offen.dev/running-offen/)
+
+---
+
+## Getting your hands dirty
+
+### Installing Offen as a systemd service
+
+We're eager to get people to install Offen and send us their feedback and tell us about their experience, so instead of adding features to the software itself, we will today look at how Offen can be installed on a Linux system that supports `systemd`.
+
+### Download the binary
+
+First, download and unpack the tarball containing latest release and verify its signature:
+
+```bash
+mkdir -p /tmp/offen-download && cd /tmp/offen-download
+curl -sSL https://get.offen.dev/v0.1.0-alpha.5 | tar -xvz
+gpg --keyserver pgp.mit.edu --recv F20D4074068C636D58B53F46FD60FBEDC90B8DA1
+gpg --verify offen-linux-amd64.asc offen-linux-amd64
+```
+
+### Install the binary
+
+Next, let's move the downloaded version to `/opt` and create a symlink in `/usr/bin` so the command is available in the system's `PATH`:
+
+```bash
+sudo mkdir -p /opt/offen/v0.1.0-alpha.5
+sudo cp offen-linux-amd64 /opt/offen/v0.1.0-alpha.5
+sudo ln -s /opt/offen/v0.1.0-alpha.5/offen-linux-amd64 /usr/bin/offen
+```
+
+You can verify that this worked as intended like this:
+
+```bash
+$ which offen
+/usr/bin/offen
+$ offen version
+INFO[0000] Current build created using revision=v0.1.0-alpha.5
+```
+
+### Scaffold the required directories
+
+Offen follows the Linux Filesystem Hierarchy Standard, so next, we need to create the directories for storing data, configuration and certificates:
+
+```bash
+sudo mkdir -p /etc/offen && sudo touch /etc/offen/offen.env
+sudo mkdir -p /var/opt/offen
+sudo mkdir -p /var/www/.cache
+```
+
+### Creating the service
+
+The Offen server needs to be run in a supervised manner so we can be sure it is always responding. `systemd` can do this for us. First we create the service definition:
+
+```bash
+sudo touch /etc/systemd/system/offen.service
+```
+
+and populate this file with the following content:
+
+```
+[Unit]
+Description=Offen Service
+
+[Service]
+ExecStart=/usr/bin/offen
+Restart=always
+
+[Install]
+WantedBy=multi-user.target
+```
+
+Now we can register the service with `systemd` itself:
+
+```bash
+sudo systemctl daemon-reload
+sudo systemctl enable offen
+sudo systemctl start offen
+```
+
+You can check whether this worked correctly using `status:`
+
+```
+$ sudo systemctl status offen
+● offen.service - Offen Service
+ Loaded: loaded (/etc/systemd/system/offen.service; enabled; vendor preset: enabled)
+ Active: active (running) since Mon 2020-01-27 15:57:58 CET; 1min ago
+ Main PID: 6701 (offen)
+ Tasks: 11 (limit: 4915)
+ CGroup: /system.slice/offen.service
+ └─6701 /usr/bin/offen
+```
+
+### Running the in-browser setup
+
+To create the initial account for your Offen install, head to `[localhost:3000/setup](http://localhost:3000/setup)` and fill out the form. After doing so, your Offen instance is ready for local use! If you want to expose this instance to the public internet, [refer to our docs site](https://docs.offen.dev/running-offen/tutorials/configuring-deploying-offen-ubuntu/) for instructions on how to get free automated SSL up and running.
+
+---
+
+## Feedback? Found a bug?
+
+If you have any feedback, comment or bug report on this milestone release, we'd love to hear from you. [Open an issue](https://github.com/offen/offen/issues) or send us an email at [hioffen@posteo.de.](mailto:hioffen@posteo.de)
diff --git a/homepage/content/articles/0060-milestone-4.md b/homepage/content/articles/0060-milestone-4.md
new file mode 100644
index 0000000..77fc055
--- /dev/null
+++ b/homepage/content/articles/0060-milestone-4.md
@@ -0,0 +1,176 @@
+title: Managing data | Offen
+description: Milestone 4 comes with enhanced account management, UX improvements and an updated demo version.
+date: 2020-06-09
+slug: managing-data
+sitemap_priority: 0.7
+sm_image_url: /theme/images/offen-blog-0060-milestone-4.jpg
+
+
+
+
+You can try using the demo yourself by running the following snippet:
+
+```bash
+curl https://demo.offen.dev | bash
+```
+
+Relevant PRs are: [367](https://github.com/offen/offen/pull/367), [346](https://github.com/offen/offen/pull/346), [347](https://github.com/offen/offen/pull/347)
+
+### UX and stats improvements
+
+An ongoing part of our work on Offen is implementing features and fixes that come from our own experience with running our own Offen instance. This is why Milestone 4 contains a few UX improvements and fixes regarding the operator facing Auditorium. Among others, we improved the referrer stats, improved the mobile UX for tabular data and fixed issues with the user flow for resetting your password.
+
+Relevant PRs are [361](https://github.com/offen/offen/pull/361), [363](https://github.com/offen/offen/pull/363), [364](https://github.com/offen/offen/pull/361https://github.com/offen/offen/pull/364),
+
+---
+
+## Next up
+
+### Accessibility Audit
+
+This week already we will have an Accessibility Audit by Stichting Accessibility. We look forward to implementing the feedback we receive and making Offen accessible for all users.
+
+Where possible we will also combine these changes with the backlog of UX improvements we are planning to implement in any case.
+
+### Security Audit
+
+Securing user data is a key aspect of Offen, so it's important to make sure we did not accidentally leave any unwanted loopholes in our system architecture. This is why Milestone 5 also includes a Security Audit by "Radically Open Security". We'll look for proper use of cryptography and a hardened HTTP interface for the server specifically, but if we can pick up other improvements along the way we won't hesitate to implement these.
+
+### Integration Test coverage
+
+Now that we have built a solid foundation for Offen, we want the public to be able to hack on and participate in the development of Offen. To make sure this is a safe and enjoyable journey, we'll add comprehensive integration test coverage for all major user stories there are so that we can always be sure the software keeps working as intended when we review and merge patches and features by others (and ourselves of course).
+
+### Getting ready for external contributions
+
+Closely related to the above, we will also do a thorough check to make sure Offen is ready for external contributions. Is documentation up to date? Does our development setup work reliably across different OSes and hardware? Is it easy to open an issue and get in touch with us? We're definitely looking forward to having the community become a part of our efforts.
+
+---
+
+## Getting your hands dirty
+
+### Adding an integration test
+
+Offen tries to be a slim and lightweight solution but nevertheless, crucial user flows can break unexpectedly and cause frustration for users, operators and developers alike. To prevent such breakages we'll focus on adding integration tests in the next milestone. In case you're curious, why not have a peek at what this looks like right now?
+
+### Testing Opt-In and Opt-Out in the context of the Auditorium
+
+Offen collects data only after opt-in. In addition to the consent banner that is shown on websites that embed Offen, the Auditorium itself allows users to manage their consent status. As an exercise, let's write a test where a user first grants consent, reviews the Auditorium and then opts out again, seeing that data has been deleted.
+
+As noted above integration tests are written using [Cypress](https://www.cypress.io/){: target="_blank"} which has a `mocha`-esque DSL for writing tests. In the `offen/offen` repository, create a new file called `integration/cypress/integration/consent.spec.js`. We're ready to write a basic test now.
+
+N.B.: these examples use `.contains('some text')` for selecting elements as this is very obvious in the context of an example. Our real world tests will use dedicated `data-testid` selectors for targeting DOM elements.*
+
+
+```jsx
+describe('Consent', function () {
+ it('displays options for opt-out and opt-in to a user without no decision set', function () {
+ cy.visit('/') // visit the index page
+ // next, we check whether both options are present and enabled
+ cy.contains('Yes please').should('exist')
+ cy.contains('Yes please').should('not.be.disabled')
+ cy.contains('I do not allow').should('exist')
+ cy.contains('I do not allow').should('not.be.disabled')
+ })
+})
+```
+
+Now that we have a basic idea of how such a test looks like, let's add interaction and check for their immediate effects on the UI:
+
+```jsx
+it('displays a link to the Auditorium after opt in only', function () {
+ cy.visit('/')
+ cy.contains('Open Auditorium').should('not.exist')
+ cy.contains('Yes please').click()
+ cy.contains('Open Auditorium').should('exist')
+})
+
+it('allows users to opt in after opt out', function () {
+ cy.visit('/')
+ cy.contains('Open Auditorium').should('not.exist')
+ cy.contains('I do not allow').click()
+ cy.contains('Open Auditorium').should('not.exist')
+ cy.contains('I do not allow').should('be.disabled')
+ cy.contains('Yes please').should('exist')
+ cy.contains('Yes please').should('not.be.disabled')
+ cy.contains('Yes please').click()
+ cy.contains('Open Auditorium').should('exist')
+})
+```
+
+Another very important feature is the ability to delete data:
+
+```jsx
+it('allows users to opt out and delete all of their data', function () {
+ cy.visit('/')
+ cy.contains('Yes please').click()
+ cy.contains('Open Auditorium').click()
+ cy.url().should('include', '/auditorium/')
+
+ cy.contains('Opt out').should('exist')
+ cy.contains('Opt out').click()
+
+ cy.contains('Opt out').should('not.exist')
+ cy.contains('Unique websites').prev('p').should('eq', '0')
+})
+```
+
+Assuming you have a running development environment set up, you can now use `make integration` to see the test runner run the tests you just added.
+
+Of course, there are a lot of other cases to cover for this feature, but you probably have an idea of how the setup works by now and how it allows us to ensure we ship working software.
+
+---
+
+## Feedback? Found a bug?
+
+If you have any feedback, comment or bug report on this milestone release, we'd love to hear from you. [Open an issue](https://github.com/offen/offen/issues) or send us an email at [hioffen@posteo.de.](mailto:hioffen@posteo.de)
diff --git a/homepage/pelicanconf.py b/homepage/pelicanconf.py
index 1e12112..2f10c9e 100644
--- a/homepage/pelicanconf.py
+++ b/homepage/pelicanconf.py
@@ -47,6 +47,15 @@ ARTICLE_SAVE_AS = 'blog/{slug}/index.html'
PLUGIN_PATHS = ['./plugins']
PLUGINS = ['decorate_content', 'assets']
+MARKDOWN = {
+ 'extension_configs': {
+ 'markdown.extensions.extra': {},
+ 'markdown.extensions.meta': {},
+ 'markdown.extensions.fenced_code': {},
+ },
+ 'output_format': 'html5',
+}
+
DECORATE_CONTENT = {
'[data-button]': ['w-100', 'w-auto-ns', 'tc', 'tl-ns', 'dib', 'mt3', 'ph4', 'pv2', 'b--solid', 'bw2'],
'[data-button-mb5]': ['w-100', 'w-auto-ns', 'tc', 'tl-ns', 'dib', 'mt3', 'mb5', 'ph4', 'pv2', 'b--solid', 'bw2'],
@@ -55,16 +64,16 @@ DECORATE_CONTENT = {
'[data-button-mb5="full"]': ['cclr-brd-black-mid', 'white', 'cclr-bg-black-mid'],
'a': ['link', 'b', 'dim'],
'a:not([data-button])': ['gray'],
+ 'h5 a': ['normal', 'moon-gray'],
'h1': ['f2', 'normal', 'lh-title', 'mt4', 'ma0', 'mb3', 'light-silver'],
'h2': ['f25', 'normal', 'lh-title', 'mt4', 'ma0', 'mb3', 'light-silver'],
'h3': ['f5', 'normal', 'mt5', 'ma0', 'mb3', '.cclr-fnt-black-mid'],
- 'h4': ['f5', 'normal', 'mt4', 'ma0', 'mb3'],
- 'h5': ['f5', 'b', 'mt4', 'ma0', 'mb1'], # text over button
+ 'h4': ['f5', 'b', 'mt4', 'ma0', 'mb1'], # text over button
+ 'h5': ['f7', 'normal', 'ma0', 'nt5', 'mb5', 'moon-gray'], # image credits
'h6': ['f5', 'lh-solid', 'normal', 'ma0', 'mb3', 'light-silver'], # date
'p': ['ma0', 'pb3'],
'blockquote': ['f5', 'i', 'ma0', 'ml4-ns', 'ml3'],
'hr': ['mt5', 'mb3', 'b--black-05'],
- '[spacer-4]': ['ma0', 'mb4']
}
DOCS_URL = 'https://docs.offen.dev'
diff --git a/homepage/theme/static/css/fix.css b/homepage/theme/static/css/fix.css
index c5663db..bd30b8f 100644
--- a/homepage/theme/static/css/fix.css
+++ b/homepage/theme/static/css/fix.css
@@ -318,9 +318,6 @@ i {
.i-on-yellow {
color: #8c7839;
}
-.inner-shaddow {
- box-shadow: inset 0px 0px 4px 0px rgba(0,0,0,0.75);
-}
/* Table */
.bg-table {
display: none;
@@ -348,6 +345,23 @@ i {
padding-right: 4rem;
}
}
+pre {
+ overflow-x: auto;
+ padding: 3rem 2rem 3rem 2rem;
+ color: #39352A;
+ background-color: #EEEEEE;
+}
+code {
+ border-radius: 3px;
+ font-size: 85%;
+ padding: 0.2em 0.4em;
+ color: #39352A;
+ background-color: #EEEEEE;
+}
+pre code {
+ padding: 0;
+}
+
/* ---------------------------------------------------
IMAGES
diff --git a/homepage/theme/static/images/offen-blog-0060-demo.gif b/homepage/theme/static/images/offen-blog-0060-demo.gif
new file mode 100644
index 0000000..6e37874
Binary files /dev/null and b/homepage/theme/static/images/offen-blog-0060-demo.gif differ
diff --git a/homepage/theme/static/images/offen-blog-0060-milestone-4.jpg b/homepage/theme/static/images/offen-blog-0060-milestone-4.jpg
new file mode 100644
index 0000000..4314e62
Binary files /dev/null and b/homepage/theme/static/images/offen-blog-0060-milestone-4.jpg differ
diff --git a/homepage/theme/templates/base.html b/homepage/theme/templates/base.html
index abb2070..208d791 100644
--- a/homepage/theme/templates/base.html
+++ b/homepage/theme/templates/base.html
@@ -134,8 +134,7 @@
- Transparent web analytics
-for everyone
+Fair web analytics
Offen is a fair alternative to common web analytics tools. Gain insights while your users have full access to their data. Self hosted, open source and free. diff --git a/homepage/theme/templates/trydemo.html b/homepage/theme/templates/trydemo.html index 7e10ab9..d047fff 100644 --- a/homepage/theme/templates/trydemo.html +++ b/homepage/theme/templates/trydemo.html @@ -18,8 +18,8 @@ Open your terminal and type
+
+
curl https://demo.offen.dev | bash
@@ -29,8 +29,8 @@
With Docker use
-
-
+
+
docker run --rm -it -p 9876:9876 offen/offen:latest demo -port 9876