Devuan testing new amprolla3 powered repository

If you like to test the new amprolla3 powered repository system here is a small list of optional /etc/apt/sources.list

deb https://pkgmaster.devuan.org/merged/ ascii main contrib non-free
deb https://pkgmaster.devuan.org/merged/ ascii-backports main contrib non-free
deb https://pkgmaster.devuan.org/merged/ ascii-proposed-updates main contrib non-free
deb https://pkgmaster.devuan.org/merged/ ascii-security main contrib non-free
deb https://pkgmaster.devuan.org/merged/ ascii-updates main contrib non-free
deb https://pkgmaster.devuan.org/devuan/ ascii-proposed main contrib non-free
deb https://pkgmaster.devuan.org/devuan/ ascii-proposed-security main contrib non-free
deb https://pkgmaster.devuan.org/merged/ ceres main contrib non-free
deb https://pkgmaster.devuan.org/devuan/ experimental main contrib non-free
deb https://pkgmaster.devuan.org/merged/ jessie main contrib non-free
deb https://pkgmaster.devuan.org/merged/ jessie-backports main contrib non-free
deb https://pkgmaster.devuan.org/devuan/ jessie-proposed main contrib non-free
deb https://pkgmaster.devuan.org/devuan/ jessie-proposed-backports main contrib non-free
deb https://pkgmaster.devuan.org/devuan/ jessie-proposed-security main contrib non-free
deb https://pkgmaster.devuan.org/merged/ jessie-proposed-updates main contrib non-free
deb https://pkgmaster.devuan.org/merged/ jessie-security main contrib non-free
deb https://pkgmaster.devuan.org/merged/ jessie-updates main contrib non-free
deb https://pkgmaster.devuan.org/devuan/ sid main contrib non-free
deb https://pkgmaster.devuan.org/merged/ stable main contrib non-free
deb https://pkgmaster.devuan.org/merged/ stable-backports main contrib non-free
deb https://pkgmaster.devuan.org/devuan/ stable-proposed main contrib non-free
deb https://pkgmaster.devuan.org/merged/ stable-proposed-updates main contrib non-free
deb https://pkgmaster.devuan.org/merged/ stable-security main contrib non-free
deb https://pkgmaster.devuan.org/merged/ stable-updates main contrib non-free
deb https://pkgmaster.devuan.org/merged/ testing main contrib non-free
deb https://pkgmaster.devuan.org/merged/ testing-backports main contrib non-free
deb https://pkgmaster.devuan.org/devuan/ testing-proposed main contrib non-free
deb https://pkgmaster.devuan.org/merged/ testing-proposed-updates main contrib non-free
deb https://pkgmaster.devuan.org/merged/ testing-security main contrib non-free
deb https://pkgmaster.devuan.org/merged/ testing-updates main contrib non-free
deb https://pkgmaster.devuan.org/merged/ unstable main contrib non-free

deb tor://devuanfwojg73k6r.onion/ should be replaced to use the onion address or you could add tor+https://pkg…. to use tor without onion addresses.

I hope this helps to end the confusion. I think some of those repositories, like sid, may be empty but that is not a 404 error, or the side that has non-free may be empty. I hope that someone will verify this list is correct before my mistake or misperception spills more panic.

PS For https and tor you need apt-transport-https & apt-transport-tor otherwise substitute http from the list, right?

PS2 The tor-project repositories are:

deb tor://sdscoq7snqtznauu.onion/torproject.org/ jessie/stretch/buster/sid main  (choose a distribution to match)
deb tor://sdscoq7snqtznauu.onion/torproject.org/ tor-experimental-0.3.2.x-stretch main (switch stretch to one you want)
or https://deb.torproject.org/torproject.org

 

PS3  Today Sunday 19th of November we discovered the following document and we are including it as a more official statement on the same subject.  This is not directed to users but to mirror masters providing them of technical details on how to set up a mirror based on the new amprolla3 official devuan repository pkgmaster.devuan.org . We hope you find it interesting.  It shows how devuan’s two repositories devuan & merged work in relation to debian repositories.

http://pkgmaster.devuan.org/devuan_mirror_walkthrough.txt

### Setting up a Devuan Package mirror -- walkthrough ###

(KatolaZ -- 20171031)

This document includes a simple walkthrough to help setting up Devuan
package mirrors. We assume that you know how to create and use an ssh 
key pair, how to use rsync, and how to setup a web server.  


0) INTRODUCTION
   
   The full Devuan package repository is available at pkgmaster.devuan.org
   All the mirrors will rsync from that main repo. The repo is contained
   in a folder "devuan/" which contains two more folders:

     merged/ : this is the folder containing the merged repo, i.e., the list
               of packages available in Devuan, where forked Devuan packages
               superseed the corresponding Debian package (if available)

     devuan/ : this folder contains the repos that are specific to Devuan,
               and the Devuan's /pool/ folder (i.e., the actual Devuan
               packages). 


   At the time of writing (Oct 2017) the full Devuan package mirror 
   requires about 8GB of space. Notice that this size is much smaller than 
   the size of a full Debian repo (which is about 2TB). The reason is that, 
   thanks to amprolla, Devuan can use all the standard Debian packages 
   (coming from a standard Debian repo) together with Devuan-specific 
   packages. This is achieved by rewriting the Filename: property of each 
   package description, and implemented on the repository side by 
   appropriately rewriting the URLs required by the client. Hence, any 
   mirror needs to provide a few rewrite rules in order to be able to serve
   Devuan packages. See SECTION 5 below for further details. 

   N.B.: The size of Devuan package repositories will most probably keep
   increasing in the future, so please take that into account when you 
   decide to host a package mirror. 

1) YOU NEED AN SSH KEY PAIR

   rsync access to pkgmaster.devuan.org is through ssh with public key.  
   You can use an existing ssh key pair or instead create an ad-hoc 
   key pair for your new Devuan mirror using:

      $ ssh-keygen
  

2) LET US KNOW ABOUT YOUR MIRROR, AND STAY INFORMED

   In order to be able to pull from pkgmaster.devuan.org, you need to have
   your ssh public key added to the list of authorised keys. Please send 
   the public ssh key you wish to use to mirrors@devuan.org, together with 
   the following information:

   - Name of the mirror contact person
   - Contact person email
   - mirror IP
   - mirror FQDN
   - Bandwidth limits (if any) 
   - Disk limits (if any)
 
   We try to guarantee a quick setup of new mirrors, and we will get back 
   to you as soon as possible. Just please be patient :)

   In addition, we require that the mirror contacts subscribe the 
   devuan-mirrors mailing list at the following address:

     devuan-mirrors@lists.dyne.org 

   This is a low-traffic mailing list where all the important announcements
   about mirror configuration and issues are posted.  
   

3) EVERYBODY: PULL!

   Once your ssh key has been set up, you should be able to rsync
   the whole package repository using:

     $ cd MIRROR_DIR
     $ rsync -avz vesta@pkgmaster.devuan.org:~/devuan ./

   where MIRROR_DIR is the directory on your mirror where the "devuan/"
   folder will be placed. By doing so, you will have the new folders:

       ${MIRROR_DIR}/devuan/devuan
       ${MIRROR_DIR}/devuan/merged
   
   The same command should be run periodically (e.g., as a cron job), and 
   normally no more than once every hour. Each mirror might be asked to set 
   slightly different update rates, if the need arises. 

   (N.B.: upon setting up your mirror, and only then, you might need to 
    run rsync twice, since amprolla3 is merging every few minutes and in 
    theory some files could have been changed during your first rsync
    of the whole stuff.)

4) WHAT USERS WILL LOOK FOR

   apt users will connect to your mirror via http/https. So you need to 
   have a web server setup at your mirror FQDN. If ${MIRROR_DIR}/devuan/
   is the ServerRoot of: 
   
      your.dev.uan.doma.in
   
   the users of your mirror will look for stuff into:

      your.dev.uan.doma.in/merged
      your.dev.uan.doma.in/devuan

   In most of the cases, Devuan is not the only thing mirrored by a mirror
   site, hence your ServerRoot might be set to the parent folder of "devuan/"
   (i.e., ${MIRROR_DIR}). In these cases, the users of your mirror will look 
   for stuff into: 

      your.dev.uan.doma.in/devuan/merged
      your.dev.uan.doma.in/devuan/devuan
   
   Your configuration might differ a bit from either of the two above. But 
   if that is the case, you know what we are talking about and you should be 
   able to find a way around it. If you are unsure, just ask (e.g., either on
   the devuan-mirrors ML, or on the devuan-dev ML, or on #devuan-dev IRC
   channel.

5) SETTING UP THE NEEDED REWRITE RULES

   Now you have to configure your webserver to rewrite some of the 
   destinations under ${MIRROR_DIR}/devuan/ so that users can find the 
   packages. There are two cases: either your mirror has also a local Debian 
   mirror under '${MIRROR_DIR}/debian/', or it does not.

5a) IF YOU HAVE A LOCAL DEBIAN MIRROR AT ${MIRROR_DIR}/debian/

    If you have a local DEBIAN mirror and it is placed at $MIRROR_DIR/debian, 
    and ${MIRROR_DIR} is your ServerRoot, then you will need to rewrite the 
    following destinations:

      /devuan/merged/pool/DEVUAN/(.*)           ->  /devuan/devuan/pool/$1
      /devuan/merged/pool/DEBIAN/(.*)           ->  /debian/pool/$1
      /devuan/merged/pool/DEBIAN-SECURITY/(.*)  ->  /debian-security/pool/$1

   For nginx, you will want to add the following section to the relevant 
   server config:

      location /devuan/merged {
                  autoindex on;
                  rewrite /devuan/merged/pool/DEVUAN/(.*)          /devuan/pool/$1;
                  rewrite /devuan/merged/pool/DEBIAN-SECURITY/(.*) /debian-security/pool/$1;
                  rewrite /devuan/merged/pool/DEBIAN/(.*)          /debian/pool/$1;
          }
 
   For apache2, you will want to use something like the following instead:

        RewriteEngine on
	RewriteRule /devuan/merged/pool/DEVUAN/(.*) /devuan/pool/$1
	RewriteRule /devuan/merged/pool/DEBIAN-SECURITY/(.*) /debian-security/pool/$1
    	RewriteRule /devuan/merged/pool/DEBIAN/(.*) /debian/pool/$1
 
   (N.B.: These rewrite rules are just indicative. If you have already a Debian 
    mirror, you know better than us where the stuff is to be found in your 
    server. If you are unsure, just shout). 
 
5b) IF YOU DON'T HAVE A LOCAL DEBIAN MIRROR

    If you don't have a local DEBIAN mirror, you will need to rewrite all the 
    DEBIAN destinations to an appropriate DEBIAN MIRROR. We assume below that
    the "devuan/" folder is directly under your ServerRoot. We also assume that 
    you choose "deb.debian.org" as Debian mirror. This is the recommended 
    choice, since "deb.debian.org" is the entry point of the Debian mirror 
    infrastructure, and should automatically redirect to the appropriate local
    Debian mirror. You need the following rewrites:

      /merged/pool/DEVUAN/(.*)           ->  /devuan/pool/$1
      /merged/pool/DEBIAN/(.*)           ->  http://deb.debian.org/debian/pool/$1
      /merged/pool/DEBIAN-SECURITY/(.*)  ->  http://deb.debian.org/debian-security/pool/$1

    In nginx you will want to add the following section to the relevant
    server config:

    location /merged {
                autoindex on;
                rewrite /merged/pool/DEVUAN/(.*)          /devuan/pool/$1;
                rewrite /merged/pool/DEBIAN-SECURITY/(.*) http://deb.debian.org/debian-security/pool/$1;
                rewrite /merged/pool/DEBIAN/(.*)          http://deb.debian.org/debian/pool/$1;
        }

   For apache, instead:

	RewriteEngine on
	RewriteRule /devuan/merged/pool/DEVUAN/(.*) /devuan/pool/$1
	RewriteRule /devuan/merged/pool/DEBIAN-SECURITY/(.*) http://deb.debian.org/debian-security/pool/$1
    	RewriteRule /devuan/merged/pool/DEBIAN/(.*) http://deb.debian.org/debian/pool/$1
 
   Again, those rewrites are just indicative. Similar instructions hold 
   for setting up https mirrors. 

If your comment is considered off-topic a new topic will be created with your comment to continue a different discussion. This community is based on open and free communication, meaning we must all respect all in minimizing the exercise of freedom to disrupt such communication. Feel free to post what you think but keep in mind the subject matter discussed. It is just as easy to start a new topic as it is to dilute the content of an existing discussion.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.