Sync via Synology NAS

This topic contains 15 replies, has 7 voices, and was last updated by  daffy 1 week ago.

  • Author
    Posts
  • March 23, 2018 at 6:26 AM #27943

    Pyromixer
    Participant

    Syncing via Synology DiskStation would be nice. Unfortunately there is no CouchDB available for Synology DiskStations.

    I found hundreds of apps for Synology. Unfortunately no CouchDB. Here is a selection of Synology Apps. Maybe with one of them the TapForms synchronization would be possible?

    Here are the original Synology apps:

    and here are a list of 3rd party apps for the synology boxes:

    March 23, 2018 at 12:27 PM #27955

    Brendan
    Keymaster

    Ya, I also have a Synology NAS and it would have been great to have CouchDB available for it.

    April 4, 2018 at 1:35 AM #28089

    Robert Marko
    Participant

    I’ve actually been researching this. I’ve made a Docker image of CouchDB, and will test the container to see if I can get a workable sync. If I can’t find a package for my Synology DSM I usually go straight to Docker Hub. I believe you can download Docker as an official package for Synology DSM. Give it a shot!

    I’ve also made a trial NextCloud container for my Bookmarks since XMarks sync is shutting down soon and the NC container is definitely persistent, even when I turn it on and off within Docker(!).

    I’m trialing WebDav sync with other iOS apps unrelated to TapForms as well for my NextCloud instance… It’s been going great so far. I’m consolidating my personal stuff and moving away from proprietary cloud systems, gradually.

    Good luck!

    April 4, 2018 at 11:26 AM #28101

    Brendan
    Keymaster

    I’d be interested to hear your experience if you manage to get a CouchDB instance running on a Synology NAS.

    April 12, 2018 at 1:44 PM #28234

    David Verdonck
    Participant

    I got CouchDB running in a Docker container on my Synology NAS for the purpose of syncing Tap Forms data across all my devices.

    It’s very simple actually, you can just install the official CouchDB Docker image from the Synology Docker UI, make sure you mount some folders on your NAS for the data and configuration volumes on your Docker container and you’re good to go.

    I also got it running via SSL on port 443 through a reverse proxy defined in the Synology Application portal, and secured with the Synology built-in Let’s Encrypt certificate system.

    Works like a charm!

    April 12, 2018 at 3:34 PM #28237

    Brendan
    Keymaster

    Hi David,

    Oh that’s great!

    I don’t think it works on my ds411slim though. Docker doesn’t show up as an installable package.

    April 13, 2018 at 12:53 AM #28244

    David Verdonck
    Participant

    Yes, you’re right, that model is not supported it seems:

    https://www.synology.com/en-global/dsm/packages/Docker

    April 21, 2018 at 2:28 PM #28334

    Carmen S.
    Participant

    Hallo David

    We also have a Synology therefore we are also very interested in this solution. Thanks for posting it :-)

    However, I have some questions… Can you help me?

    It’s very simple actually, you can just install the official CouchDB Docker image from the Synology Docker UI, make sure you mount some folders on your NAS for the data and configuration volumes on your Docker container and you’re good to go.

    We have done this.

    I also got it running via SSL on port 443 through a reverse proxy defined in the Synology Application portal, and secured with the Synology built-in Let’s Encrypt certificate system.

    Can you specify this?
    We installed Dockers and CouchDB and now we don’t know what to do.

    Thank you!!

    Carmen

    April 21, 2018 at 4:22 PM #28340

    Brendan
    Keymaster

    Hi Carmen,

    I don’t have experience with the Docker thing, but there may be useful information for you in the CouchDB chapter of the online user manual here:

    https://www.tapforms.com/help-mac/5/en/topic/couchdb-sync

    March 23, 2019 at 12:29 AM #34078

    ELLIOTTCABLE
    Participant

    Since I came across this early-on when preparing to set all this up, and it looks like a few people have had trouble with this, I thought I’d apply some of my knowledge and leave a resource for future Googlers.

    First off, some links from which I pieced all of this together:

    ~~ Prerequisites ~~

    First off, I’m assuming you already have Synology’s dynamic DNS aka ‘DDNS’ (tutorial), a custom domain-name (tutorial), and a Let’s Encrypt SSL certificate (tutorial) setup. If you haven’t done those, do them! They’ll make your life easier not just for Tap Forms, but for … basically everything. Very worth your time (and a couple bucks if you don’t already have a domain.)

    ~~ Installing Docker and CouchDB ~~

    First off, visit the ‘Package Center’ to install Docker and Synology’s Docker ‘app’:

    Once it’s installed, open it from the Main Menu, in the top left. The overview you’ll be presented with is pretty useless, as you haven’t downloaded any “images” (templates from which you can create apps, basically), nor started any “containers” (said apps) yet.

    There’s two images we’re going to want to install and configure: the CouchDB server itself, and the official GUI CouchDB management tool known as ‘Fauxton.’ Go ahead and select the ‘Registry’ tab on the left in the Docker app:

    You’re going to download those two images here. For each of these, use the search field to find the image, right-click it, and then click “Download this image.” When asked to choose a version, accept the default of “latest.”

    Repeat this for both these image names:

    • couchdb
    • 3apaxicom/fauxton

    When they’re both downloaded, click ‘Images’ in the sidebar, and verify that both of the above are listed.

    ~~ Starting CouchDB ~~

    Next, we’re going to create an actual container out of the CouchDB image — an instance of the app, configured and running on your Synology.

    In that ‘Images’ section of the Docker app, select the CouchDB image, and then click ‘Launch’ to bring up Synology’s container-wizard:

    Give the container a unique name, and feel free to limit the available resources; then click ‘Advanced Settings:’

    Configure the following:

    1. Enable ‘auto-restart,’ to ensure Tap Forms can always sync.
    2. Create a folder on your NAS for the CouchDB server’s data and configuration. I used <volume>/docker/tapforms/ as the root, for instance; and you’ll need several subdirectories, as described below.

      Within that root, you’re then going to add add Docker volume mappings to each of the CouchDB directories:

      • <volume>/docker/tapforms/etc, mounted to the image’s /opt/couchdb/etc/local.d
      • <volume>/docker/tapforms/data, mounted to the image’s /opt/couchdb/data
      • <volume>/docker/tapforms/logs (plural), mounted to the image’s /opt/couchdb/log (singular)

      For each of the above (under the ‘Volumes’ tab of ‘advanced settings’ while creating the container) you’ll have to click ‘Add Folder’, and select the first path — on your NAS volume — from a tree:

      Then, you’ll have to click the ‘Mount path’ field, and type the second path — the one inside the container — into the field:

    3. Map the Docker container’s CouchDB port to the NAS’s local, internal CouchDB port.

      Switch to the ‘Port Settings’ tab. Click ‘Auto’ next to the entry for 5984, and change that to also be 5984:

    Finally, click ‘Apply’ to close the advanced settings, and then click ‘Next.’ You should see a summary of the settings (the super-important volume-mount paths won’t be listed, so watch out.) Click ‘Apply’ again:

    Synology will now boot the container in the background; so let’s go setup the other one while we wait.

    ~~ Starting Fauxton ~~

    Repeat the above, but for the 3apaxicom/fauxton container. Give it a unique name, and limit the resources appropriately. Tell it to create a DSM desktop shortcut, at the port 8006 — that’s not a typo, we’ll get it all hooked up together later:

    Change the Synology-local port from ‘Auto’ to 8005; (again, not a typo):

    Now, for a weird bit: you need to create a ‘link’ between the containers. This is a secure, secret port, basically. Click the ‘Links’ tab of the advanced settings, and click the ‘+’ to add one. Select your CouchDB container; and then type db as the alias:

    Finally, change the ‘Execution Command’ under ‘Environment’ to exactly this command, quote-marks and everything: sh -c 'fauxton -c http://$DB_PORT_5984_TCP_ADDR:$DB_PORT_5984_TCP_PORT'

    Apply, Next, double-check, Apply. Boom! Both containers should be running. Click over to the ‘Containers’ tab in the Docker-app’s sidebar to double-check:

    ~~ Configuring port-forwarding, SSL, and reverse-proxying ~~

    Phew! We’re halfway there. Let’s take a break from Docker, and double-check some Synology DSM settings that are really important. First, Synology’s built-in reverse proxy — this will ‘hide’ all our shmancy CouchDB passwords and stuff behind secure HTTPS / TLS encryption.

    Twice, we’re gonna: open ‘Control Panel,’ ‘Application Portal’ (yeah, I don’t get the name either), ‘Reverse Proxy.’ Click ‘Create.’

    You’re gonna route incoming requests from (confusingly called ‘Source’) HTTPS, on your domain-name, port 6984, to (called ‘Destination’) HTTP on localhost, port 5984:

    Hit ‘OK,’ then repeat for the web-GUI we installed — this time, route incoming requests from HTTPS, on your domain-name, port 8006, to HTTP on localhost, port 8005. You can also enable HSTS and HTTP/2 on this one.

    Then, you’ll need to configure your router’s port-forwarding. Luckily, Synology is absurdly badass, and can do most of that for most of us — as long as your router has something called UPNP. Synology has rather thorough documentation on this, but at least for me, it was about this simple: open ‘Control Panel,’ ‘External Access,’ ‘Router Configuration.’ Click the ‘Set up router’ button at the top.

    Then, ‘Create’ some rules, and select ‘Built-in application.’ Do not select the Docker entries — select, instead, the Reverse Proxy entries you just created for ports 8006 and 6984; then hit ‘Apply.’

    (You may have to do this manually, on your router, instead. I’m afraid I can’t help you there!)

    ~~ Configuring CouchDB ~~

    Let’s verify that everything’s routed properly. Open https://yourdomain.blah:8006 in a new tab (you should be able to just click the icon on the DSM homescreen, too, if you set that up above.)

    If Fauxton loads, then head on over to the ‘Verify’ tab on the left, and click ‘Verify installation.’ Everything should succeed (except maybe the replication section? Not sure what’s up with that, but it consistently fails for me.)

    Next, we need to lock down the installation. Right now, your CouchDB is running in “Admin Party” mode — which is a cute way of saying ‘horribly insecure.’ Literally everybody on the Internet currently has admin access to your database, oh no!

    Let’s fix that. Click ‘Setup’ in the sidebar, then ‘Configure a Single Node.’

    You’ll be presented with a form via which you can add what is effectively a root password to your CouchDB server. (Note that this is *not* your personal non-admin user-account! We’ll create that later.) Choose a secure root password, write it down somewhere (may I heartily suggest 1Password? 🤣), and create your administration account. Finally, enter :: (exactly two colons, no spacing) as the ‘bind address’, and leave the port alone.

    Once you’ve submitted that, reload the page — Fauxton won’t be able to successfully make any more changes to your CouchDB installation until you’re logged-in under that root account. Enter the information you just saved, and hit ‘Log In.’

    Switch over to the ‘Config’ sidebar entry, and let’s tweak some configuration values:

    1. Set [chttpd] require_valid_user to true
    2. Set [log] file to /opt/couchdb/log/couch.log
    3. Set [log] writer to file

    Now — before you close the page! — you need to make sure Fauxton can still authenticate with CouchDB. This isn’t very clear, and took me ages of searching, but it turns out you need to *add* a config-option. Click ‘Add Option’ at the top of the page, and enter the following three values, precisely: httpd, WWW-Authenticate, and Basic realm="administrator" — spaces, quotes, and all. Then click ‘Create.’

    The very last step of configuration there, is to create a non-admin user for Tap Forms to authenticate as. (This is optional, but a good idea in general.)

    Click over to the ‘Databases’ tab in the sidebar, and select the _users database. Hit ‘Create Document.’

    Delete everything in the text-editor that appears, and replace it with the snippet from this Stack Overflow answer, changing your desired user-name (in two places!) and password as appropriate:

    Click ‘Create Document.’ (Note that CouchDB will hash up that password you enter and store it safely; you’ll never be able to view it again. Store it in 1Password, too!)

    Now that you’ve done all that, reboot the CouchDB container so that the new configuration will take effect. Return to DSM, open the Docker app’s ‘Containers’ tab, right-click on your CouchDB container, and select the ‘Restart’ action:

    If you’re Terminal-savvy, you can give your server a real quick test: run curl https://ds.ell.io:6984/, replacing the domain with the one you’ve set up for your DiskStation or your Synology.me DDNS domain. If you’ve got everything working, you will get an error — that’s what we want! Something along the lines of {"error":"unauthorized","reason":"Authentication required."}.

    ~~ Configuring Tap Forms ~~

    You’re in the home stretch! I know it’s been a slog, but we’re, finally, almost done.

    The very last step is to configure Tap Forms itself on your devices. Brendan covers it pretty well, so I’ll just add some addenda and errata to his writeup here:

    • Ignore ‘Installing CouchDB’ and ‘Setup,’ we just did all that …
    • … except for the notes on admin accounts. Basically, you should sign into the root account we created earlier inside a Tap Forms app at least once, so it can create your db-deadb33f... databases. Then you can tap sign out, and sign back in using the second, non-administrator account you created.
    • Pretty much everything about SSL can be ignored: Synology’s reverse-proxy handled this for us.
    • It was unclear to me, and maybe also to you … so note that all of the above is specific to a single ‘database’ in Tap Forms. There’s no way to configure Tap Forms as a whole to sync all of its databases via CouchDB, and you’ll have to duplicate the setup for every new database you create.

    Hope this was helpful to someone! Maybe I shouldn’t have spent like six hours writing it up, yikes. I’ll be ‘following’ this thread via WordPress’s settings, so I’ll get an e-mail if you have any questions — though I’m not sure how much help I can offer, as I’m definitely not an expert on any of the above!

    March 23, 2019 at 12:43 AM #34083

    ELLIOTTCABLE
    Participant

    I think my posts to this thread are getting spam-filtered. @Brendan, can you restore my original post? It has, uh, a … lot of links and images, so no wonder. ¯\_(ツ)_/¯

    March 23, 2019 at 1:35 PM #34086

    Brendan
    Keymaster

    I’ll look into it.

    March 23, 2019 at 1:41 PM #34088

    Brendan
    Keymaster

    I restored two posts that were marked as spam, but they were both the same, so I deleted one of them.

    That’s really great information!

    Thanks very much for sharing Elliott!

    April 16, 2019 at 9:20 AM #34422

    daffy
    Participant

    Hi, this is in reply to the detailed help provided by @elliottcable above.

    I am wanting to use CouchDB on my synology drive because I’m having trouble with the iCloud sync.

    My concern is that I am not quite techie enough to do it. I did manage to set up the NAS drive, but it took a lot of reading and testing (and patience!).

    How user friendly is the system you have described? Would I need specialist knowledge to enable me to set it up?

    I have a domain name I can use and I understand SSl enough to be able to set it up (I think), but I’m unsure about the DDNS instructions you linked to.
    https://www.synology.com/en-us/knowledgebase/DSM/help/DSM/AdminCenter/connection_ddns
    I thought it would be something I needed to install on my NAS, but perhaps I misunderstood. I find the Synology ‘help’ files often miss out crucial info that would enable someone with slightly less tech knowledge to be able to implement their instructions.

    I’d very much appreciate your thoughts on how achievable this might be for me. If it’s way above my knowledge and I can’t learn enough by reading up, then perhaps it’s too much for me to embark on.

    Cheers,
    Nicky

    April 17, 2019 at 3:43 AM #34429

    ELLIOTTCABLE
    Participant

    Hey there!

    1. The process is definitely long, but, and this is key, not necessarily particularly arcane, as these things go. That is, each of the steps or components of the process I detailed above, is a well-known / well-documented Thing That You Can Google; those ‘TTYCGs’ are mostly an issue of figuring out the right words to search for, and I hope you can glean some of those from my instructions if you run into any problems!
    2. DDNS is a good example: if you google “dyndns” or “dynamic DNS”, you can find a wealth of information. It won’t be instant or trivial, but you will probably be able to relate most of what you find back to either the Synology docs, the Synology interface buttons ‘n widgets ‘n doodles, or at worst, the tutorial I link above (note that I linked an external tutorial for each of those topics; I, too, thought the Synology docs weren’t particularly beginner-friendly.) In the specific case of DDNS, don’t worry too much — basically select the Synology option, register a free account, and setup a daffy.synology.me or something domain right there in the Synology settings once you’re registered.
    3. Finally, a general thought, if you’re largely not technical: simultaneously the best, and worst, thing about tech? No matter how complicated it is, you can always get there. It just might take a while. The thing to note is that it took me most of an afternoon and evening to figure out (well, and document … but still.) all of the above, and I’m very experienced with a lot of the various moving components. I strongly feel that you could figure it out, too, whatever your background — but do be prepared to spend a lot of time Googling things, banging your head against poor documentation, or writing up questions on the Internet! If “many hours invested” doesn’t sound like a good trade-off for this, that’s when to go ‘hm, is this really worth my time.’

    tl;dr you can do it! it might take a while tho. Good luck!

    April 17, 2019 at 3:10 PM #34433

    daffy
    Participant

    Wow, thank you so much for such a thoughtful and detailed reply. Much appreciated.
    I do feel optimistic about giving it a go. Setting up the NAS drive itself took several head bangs against the wall, lots of swearing and waking away (sometimes for several days!) before coming back and trying again, but I did it and it felt great.

    I think the key for me will be choosing the right time to look at it. I have a lot on at the minute and if I let this distract me I will probably get more frustrated with it because I know I should be spending my time elsewhere.

    Thanks again for the info. I will certainly be trying it out. :)

You must be logged in to reply to this topic.