#144: f23 atomic iso configures docker loopback storage -------------------------+-------------------- Reporter: jasonbrooks | Owner: Type: defect | Status: new Priority: major | Milestone: Future Component: --- | Keywords: -------------------------+-------------------- ISO installs of f23 atomic get configured w/ loopback storage, which is a bad thing: http://www.projectatomic.io/blog/2015/06/notes-on-fedora- centos-and-docker-storage-drivers/.
F22 Atomic had the correct config.
Running "docker info" on a VM installed from Fedora-Cloud_Atomic- x86_64-22.iso yields:
{{{ ... Storage Driver: devicemapper Pool Name: fedora-docker--pool Pool Blocksize: 65.54 kB Backing Filesystem: extfs Data file: Metadata file: ... }}}
Running "docker info" on a VM installed from Fedora-Cloud_Atomic- x86_64-23.iso yields:
{{{ ... Storage Driver: devicemapper Pool Name: docker-253:0-153054-pool Pool Blocksize: 65.54 kB Backing Filesystem: extfs Data file: /dev/loop0 Metadata file: /dev/loop1 ... }}}
I looked over the kickstarts, but I'm not clear on what might be causing this.
On Wed, Dec 9, 2015 at 12:26 PM, Fedora Cloud Trac Tickets cloud-trac@fedoraproject.org wrote:
#144: f23 atomic iso configures docker loopback storage I looked over the kickstarts, but I'm not clear on what might be causing this. Ticket URL: https://fedorahosted.org/cloud/ticket/144
I'm replying to list to avoid cluttering the ticket. I'm in the vicinity of a related issue, trying to figure out what's doing the setup, because I didn't use the prescribed automatic partitioning layout, therefore don't have any lvmthinp stuff setup yet. The ISO seems to create a system that's somehow making assumptions, but I don't know where it's getting that info.
For example, I'm running into this:
● docker-storage-setup.service loaded failed failed Docker Storage Setup ● docker.service loaded failed failed Docker Application Container Engine
The cause for storage setup service failing is Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Volume group "sda2" not found Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Cannot process volume group sda2 Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Metadata volume docker-poolmeta already exists. Not creating a new one. Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Please provide a volume group name Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Run `lvcreate --help' for more information.
But where is it thinking there'd be a VG called sda2?
OK so I do [chris@f23a ~]$ cat /usr/lib/systemd/system/docker-storage-setup.service [Unit] Description=Docker Storage Setup After=cloud-final.service Before=docker.service
[Service] Type=oneshot ExecStart=/usr/bin/docker-storage-setup EnvironmentFile=-/etc/sysconfig/docker-storage-setup
[Install] WantedBy=multi-user.target
That suggests looking at /etc/sysconfig/docker-storage-setup, which does not exist (yet?). I also don't know the significance of the - right after = in that line. There is a /etc/sysconfig/docker-storage file that contains:
# This file may be automatically generated by an installation program.
# By default, Docker uses a loopback-mounted sparse file in # /var/lib/docker. The loopback makes it slower, and there are some # restrictive defaults, such as 100GB max storage.
# If your installation did not set a custom storage for Docker, you # may do it below.
# Example: Use a custom pair of raw logical volumes (one for metadata, # one for data). # DOCKER_STORAGE_OPTIONS = --storage-opt dm.metadatadev=/dev/mylogvol/my-docker-metadata --storage-opt dm.datadev=/dev/mylogvol/my-docker-data
DOCKER_STORAGE_OPTIONS=
So it might be that the Fedora 23 Atomic ISO behavior is the result of upstream behavior, and Fedora 22 had a modifier that no longer exists in Fedora 23?
On 12/09/2015 07:20 PM, Chris Murphy wrote:
On Wed, Dec 9, 2015 at 12:26 PM, Fedora Cloud Trac Tickets > cloud-trac@fedoraproject.org wrote: >> #144: f23 atomic iso
configures docker loopback storage >> I looked over the kickstarts, but I'm not clear on what might be causing >> this. >> Ticket URL: https://fedorahosted.org/cloud/ticket/144 > > I'm replying to list to avoid cluttering the ticket. I'm in the > vicinity of a related issue, trying to figure out what's doing the > setup, because I didn't use the prescribed automatic partitioning > layout, therefore don't have any lvmthinp stuff setup yet. The ISO > seems to create a system that's somehow making assumptions, but I > don't know where it's getting that info. > > For example, I'm running into this: > > ● docker-storage-setup.service loaded failed failed Docker > Storage Setup > ● docker.service loaded failed failed > Docker Application Container Engine > > The cause for storage setup service failing is > Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Volume > group "sda2" not found > Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Cannot > process volume group sda2 > Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Metadata > volume docker-poolmeta already exists. Not creating a new one. > Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Please > provide a volume group name > Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Run > `lvcreate --help' for more information. > > But where is it thinking there'd be a VG called sda2? > > OK so I do > [chris@f23a ~]$ cat /usr/lib/systemd/system/docker-storage-setup.service > [Unit] > Description=Docker Storage Setup > After=cloud-final.service > Before=docker.service > > [Service] > Type=oneshot > ExecStart=/usr/bin/docker-storage-setup > EnvironmentFile=-/etc/sysconfig/docker-storage-setup > > [Install] > WantedBy=multi-user.target > > > That suggests looking at /etc/sysconfig/docker-storage-setup, which > does not exist (yet?). I also don't know the significance of the - > right after = in that line. There is a /etc/sysconfig/docker-storage > file that contains: > > # This file may be automatically generated by an installation program. > > # By default, Docker uses a loopback-mounted sparse file in > # /var/lib/docker. The loopback makes it slower, and there are some > # restrictive defaults, such as 100GB max storage. > > # If your installation did not set a custom storage for Docker, you > # may do it below. > > # Example: Use a custom pair of raw logical volumes (one for metadata, > # one for data). > # DOCKER_STORAGE_OPTIONS = --storage-opt
dm.metadatadev=/dev/mylogvol/my-docker-metadata --storage-opt >
dm.datadev=/dev/mylogvol/my-docker-data > > DOCKER_STORAGE_OPTIONS= > >
So it might be that the Fedora 23 Atomic ISO behavior is the result
of > upstream behavior, and Fedora 22 had a modifier that no longer exists > in Fedora 23?
You can populate /etc/sysconfig/docker-storage-setup and have it do what you want. The thing about it is that it isn't the smartest script and if you don't tell it exactly what you want it to do, it can go off the rails pretty easily (especially for a storage setup it isn't used to seeing).
man docker-storage-setup will show you want you can put in the file.
I'm guessing for your setup docker-storage-setup is getting confused and things sda2 is a VG and tries to do something with that.. Probably a bug that needs to be filed.
Dusty
On Wed, Dec 9, 2015 at 7:53 PM, Dusty Mabe dusty@dustymabe.com wrote:
You can populate /etc/sysconfig/docker-storage-setup and have it do what you want. The thing about it is that it isn't the smartest script and if you don't tell it exactly what you want it to do, it can go off the rails pretty easily (especially for a storage setup it isn't used to seeing).
man docker-storage-setup will show you want you can put in the file.
Yeah this doesn't work for me either, I still get an error. I included it in the bug below.
I'm guessing for your setup docker-storage-setup is getting confused and things sda2 is a VG and tries to do something with that.. Probably a bug that needs to be filed.
https://bugzilla.redhat.com/show_bug.cgi?id=1290283
On 12/09/2015 11:52 PM, Chris Murphy wrote:
On Wed, Dec 9, 2015 at 7:53 PM, Dusty Mabe dusty@dustymabe.com wrote: > >> You can populate /etc/sysconfig/docker-storage-setup and have it
do what you >> want. The thing about it is that it isn't the smartest script and if you >> don't tell it exactly what you want it to do, it can go off the rails >> pretty easily (especially for a storage setup it isn't used to seeing). >> >> man docker-storage-setup will show you want you can put in the file. > > Yeah this doesn't work for me either, I still get an error. I included > it in the bug below. > >> I'm guessing for your setup docker-storage-setup is getting confused >> and things sda2 is a VG and tries to do something with that.. >> Probably a bug that needs to be filed. > > https://bugzilla.redhat.com/show_bug.cgi?id=1290283
Since LVM isn't used docker-storage-setup looks like it is just exiting out and not doing anything (but the messages from the logs don't clearly indicate what is happening). A question is: what does your system storage look like before docker-storage-setup runs and what do you want it to look like after?
If I know that info maybe I can help you get there.
Dusty
On Wed, Dec 9, 2015 at 10:06 PM, Dusty Mabe dusty@dustymabe.com wrote:
On 12/09/2015 11:52 PM, Chris Murphy wrote:
On Wed, Dec 9, 2015 at 7:53 PM, Dusty Mabe dusty@dustymabe.com wrote: >
You can populate /etc/sysconfig/docker-storage-setup and have it do what
you >> want. The thing about it is that it isn't the smartest script and if you >> don't tell it exactly what you want it to do, it can go off the rails
pretty easily (especially for a storage setup it isn't used to seeing).
man docker-storage-setup will show you want you can put in the file. >
Yeah this doesn't work for me either, I still get an error. I included >
it in the bug below. > >> I'm guessing for your setup docker-storage-setup is getting confused >> and things sda2 is a VG and tries to do something with that.. >> Probably a bug that needs to be filed. > > https://bugzilla.redhat.com/show_bug.cgi?id=1290283
Since LVM isn't used docker-storage-setup looks like it is just exiting out and not doing anything (but the messages from the logs don't clearly indicate what is happening). A question is: what does your system storage look like before docker-storage-setup runs and what do you want it to look like after?
If I know that info maybe I can help you get there.
I was just trying to resolve the confusing error message. But even after satisfying it by creating a thin pool, I still get the same "sda2" error message. I've updated the bug.
Until the docker patch mentioned at the tail end of this bug lands, it seems a bit of a hassle to use Btrfs as backing. https://github.com/docker/docker/issues/7952
On Wed, Dec 09, 2015 at 05:20:28PM -0700, Chris Murphy wrote:
On Wed, Dec 9, 2015 at 12:26 PM, Fedora Cloud Trac Tickets cloud-trac@fedoraproject.org wrote:
#144: f23 atomic iso configures docker loopback storage I looked over the kickstarts, but I'm not clear on what might be causing this. Ticket URL: https://fedorahosted.org/cloud/ticket/144
I'm replying to list to avoid cluttering the ticket. I'm in the vicinity of a related issue, trying to figure out what's doing the setup, because I didn't use the prescribed automatic partitioning layout, therefore don't have any lvmthinp stuff setup yet. The ISO seems to create a system that's somehow making assumptions, but I don't know where it's getting that info.
For example, I'm running into this:
● docker-storage-setup.service loaded failed failed Docker Storage Setup ● docker.service loaded failed failed Docker Application Container Engine
The cause for storage setup service failing is Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Volume group "sda2" not found Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Cannot process volume group sda2 Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Metadata volume docker-poolmeta already exists. Not creating a new one. Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Please provide a volume group name Dec 09 16:57:46 f23a.localdomain docker-storage-setup[759]: Run `lvcreate --help' for more information.
But where is it thinking there'd be a VG called sda2?
OK so I do [chris@f23a ~]$ cat /usr/lib/systemd/system/docker-storage-setup.service [Unit] Description=Docker Storage Setup After=cloud-final.service Before=docker.service
[Service] Type=oneshot ExecStart=/usr/bin/docker-storage-setup EnvironmentFile=-/etc/sysconfig/docker-storage-setup
[Install] WantedBy=multi-user.target
That suggests looking at /etc/sysconfig/docker-storage-setup, which does not exist (yet?). I also don't know the significance of the - right after = in that line. There is a /etc/sysconfig/docker-storage file that contains:
# This file may be automatically generated by an installation program.
# By default, Docker uses a loopback-mounted sparse file in # /var/lib/docker. The loopback makes it slower, and there are some # restrictive defaults, such as 100GB max storage.
# If your installation did not set a custom storage for Docker, you # may do it below.
# Example: Use a custom pair of raw logical volumes (one for metadata, # one for data). # DOCKER_STORAGE_OPTIONS = --storage-opt dm.metadatadev=/dev/mylogvol/my-docker-metadata --storage-opt dm.datadev=/dev/mylogvol/my-docker-data
DOCKER_STORAGE_OPTIONS=
So it might be that the Fedora 23 Atomic ISO behavior is the result of upstream behavior, and Fedora 22 had a modifier that no longer exists in Fedora 23?
--
How does this turn out?
I hope ok
#144: f23 atomic iso configures docker loopback storage -------------------------+--------------------- Reporter: jasonbrooks | Owner: Type: defect | Status: new Priority: major | Milestone: Future Component: --- | Resolution: Keywords: | -------------------------+---------------------
Comment (by jasonbrooks):
https://github.com/projectatomic/fedora-productimg-atomic is responsible for setting this up correctly. There's a bug (https://bugzilla.redhat.com/show_bug.cgi?id=1290257) and pending fixed pkg pending... not sure if it'll fix this issue as well.
#144: f23 atomic iso configures docker loopback storage -------------------------+--------------------- Reporter: jasonbrooks | Owner: Type: defect | Status: new Priority: major | Milestone: Future Component: --- | Resolution: Keywords: | -------------------------+---------------------
Comment (by jasonbrooks):
This pkg fixes my issue: https://bodhi.fedoraproject.org/updates/FEDORA-2015-1b6b8653f0
You can test w/ https://bugzilla.redhat.com/show_bug.cgi?id=1290257#c4 and give it karma
#144: f23 atomic iso configures docker loopback storage -------------------------+--------------------- Reporter: jasonbrooks | Owner: Type: defect | Status: new Priority: major | Milestone: Future Component: --- | Resolution: Keywords: | -------------------------+---------------------
Comment (by jzb):
Looks good here too. Assuming this gets pulled in for the two-week due next week, we're good. Woot.
#144: f23 atomic iso configures docker loopback storage -------------------------+--------------------- Reporter: jasonbrooks | Owner: Type: defect | Status: closed Priority: major | Milestone: Future Component: --- | Resolution: fixed Keywords: | -------------------------+--------------------- Changes (by jasonbrooks):
* status: new => closed * resolution: => fixed
cloud@lists.stg.fedoraproject.org
-
Chris Murphy -
Dusty Mabe -
Fedora Cloud Trac Tickets