[geany/www.geany.org] 1e16f2: Add Dockerfile and documentation for local development
Enrico Tröger
git-noreply at xxxxx
Sun Jul 14 22:19:19 UTC 2019
Branch: refs/heads/master
Author: Enrico Tröger <enrico.troeger at uvena.de>
Committer: Enrico Tröger <enrico.troeger at uvena.de>
Date: Sun, 14 Jul 2019 22:19:19 UTC
Commit: 1e16f26d23ca3a5599a62ebd2f3d1a99f7980b65
https://github.com/geany/www.geany.org/commit/1e16f26d23ca3a5599a62ebd2f3d1a99f7980b65
Log Message:
-----------
Add Dockerfile and documentation for local development
Alternatively to setting up a virtualenv and a local database, we
provide a Dockerfile to create a container image to be ran locally
for easier setup.
Quick usage: make docker-build && make docker-run
Modified Paths:
--------------
.dockerignore
.gitignore
Makefile
README.dev.md
docker/Dockerfile
docker/entrypoint.sh
docker/irc_userlist
docker/local_settings.docker.py
geany/settings.py
tox.ini
Modified: .dockerignore
43 lines changed, 43 insertions(+), 0 deletions(-)
===================================================================
@@ -0,0 +1,43 @@
+**/*.pyc
+**/__pycache__
+*.egg
+*.egg-info
+*.pyc
+*.pyd
+*.pyo
+*.swp
+.Python
+.cache
+.coverage
+.coverage
+.coverage.*
+.dockerignore
+.eggs
+.git
+.gitignore
+.tox
+.venv
+build
+dist
+docker/data
+docs/_build
+env
+venv
+
+
+# MacOS / Windows stuff
+.DocumentRevisions-V100
+.fseventsd
+.Spotlight-V100
+.TemporaryItems
+.Trashes
+.VolumeIcon.icns
+.com.apple.timemachine.donotpresent
+.DS_Store
+.AppleDouble
+.LSOverride
+.AppleDB
+.AppleDesktop
+Network Trash Folder
+Temporary Items
+.apdisk
Modified: .gitignore
1 lines changed, 1 insertions(+), 0 deletions(-)
===================================================================
@@ -4,3 +4,4 @@
local_settings.py
venv
geany/media
+docker/data
Modified: Makefile
38 lines changed, 38 insertions(+), 0 deletions(-)
===================================================================
@@ -0,0 +1,38 @@
+# -*- coding: utf-8 -*-
+#
+# LICENCE: This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+
+docker-run:
+ docker run \
+ --rm \
+ --interactive=true \
+ --tty=true \
+ --user "$$(id --user):$$(id --group)" \
+ --mount "type=bind,src=$$(pwd),dst=/app" \
+ --publish 8000:8000 \
+ --name geany_dev \
+ geany_dev:latest
+
+
+docker-build:
+ docker build \
+ --file docker/Dockerfile \
+ --tag geany_dev \
+ .
+
+
+docker-clean:
+ rm -rf docker/data
Modified: README.dev.md
87 lines changed, 70 insertions(+), 17 deletions(-)
===================================================================
@@ -15,8 +15,18 @@ extra packages installed on your system.
The code requires Python 3.5 or higher.
-Prepare your system
--------------------
+Get the code
+------------
+
+Perform a usual clone of the www.geany.org repository from Github:
+
+ git clone git://github.com/geany/www.geany.org
+
+
+Local setup using virtualenv
+----------------------------
+
+### Prepare your system
To ease package handling and to get a clean environment, we use
virtualenv. Into the virtualenv we will install Django, Mezzanine and a couple of
@@ -31,16 +41,7 @@ packages:
apt-get install python3-venv python3-pip python3-dev build-essential libmysqlclient-dev libmemcached-dev
-Get the code
-------------
-
-Perform a usual clone of the www.geany.org repository from Github:
-
- git clone git://github.com/geany/www.geany.org
-
-
-Setting up a virtualenv
------------------------
+### Setting up a virtualenv
Change into the freshly cloned repository directory and execute the following commands
to create a new virtualenv and install required Python packages:
@@ -55,8 +56,7 @@ This will setup a new virtualenv, upgrade the Python package manager
pip and install required packages for the website.
-Create a local config
----------------------
+### Create a local config
Use a text editor of choice (we all know what this would be...) and create a new file
*local_settings.py* in *www.geany.org/geany/* (next to the existing *settings.py*).
@@ -132,8 +132,7 @@ The database dump contains a default admin user:
password: change-me
-Start the development server
-----------------------------
+### Start the development server
After you set up everything as described above, you are ready
to start the development server to actually do something:
@@ -152,8 +151,62 @@ to reload the changed file(s). This is very helpful.
To stop the server, simply interrupt it with *Ctrl-C*.
+Local setup using Docker
+------------------------
+
+Alternatively, a Dockerfile is provided to build a Docker image
+and to run the website in a Docker container.
+This is the easiest way to get a local environment running.
+
+### Local config
+
+When using the Docker image, a prepared local settings file is
+used with already adjusted settings for running in a Docker container.
+This file is located at `docker/local_settings.docker.py`.
+
+
+### Build container image
+
+First, you need to build the image:
+
+ make docker-build
+
+This will take some time but is only necessary once.
+
+Note: before building the image, carefully review the Dockerfile
+and especially make sure you use a base image you can trust.
+
+
+### Start the container
+
+After the image is built, you can start the container:
+
+ make docker-run
+
+On the first run, the database is setup, screenshots are downloaded to be
+locally available as well as a few more preparations.
+
+Once running, you can open the resulting site in your browser by pointing it
+to http://localhost:8000.
+
+Basically now you are done and you can start improving the website.
+A little detail you might notice: once you change any .py file
+which is knwon by Django, the development server will restart automatically
+to reload the changed file(s). This is very helpful.
+
+To stop the container, simply interrupt it with *Ctrl-C*.
+
+### Cleanup
+
+All files created on first container startup are stored in `docker/data`.
+To start from scratch (e.g. with a fresh database, no uploads, etc.), simply
+delete this directory or run:
+
+ make docker-clean
+
+
Management Commands
------------------------
+-------------------
In addition to the usual Django management commands (for a
list run `python manage.py`), the Geany apps provide a few more.
Modified: docker/Dockerfile
101 lines changed, 101 insertions(+), 0 deletions(-)
===================================================================
@@ -0,0 +1,101 @@
+# LICENCE: This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+#
+# Docker container to run the Geany website locally for development.
+# This is not intended for production!!
+# Before using this image, please get yourself familiar with Docker and
+# carefully review this Dockerfile before using it!
+#
+# Build image:
+# docker build -f docker/Dockerfile -t geany_dev .
+# Run container:
+# docker run --rm -it --user "$(id --user):$(id --group)" --mount "type=bind,src=$(pwd),dst=/app" -p 8000:8000 --name geany_dev geany_dev:latest
+
+# Intermediate build container
+FROM debian:stretch-slim AS builder
+
+ENV LANG=C.UTF-8 \
+ DEBIAN_FRONTEND=noninteractive \
+ # Extra python env
+ PYTHONDONTWRITEBYTECODE=1 \
+ PYTHONUNBUFFERED=1 \
+ PIP_DISABLE_PIP_VERSION_CHECK=1 \
+ PIP_FORMAT="columns" \
+ PIP_NO_BINARY=":all:" \
+ PIP_CACHE_DIR="/tmp/pip" \
+ PIP_TIMEOUT=60
+
+RUN apt-get update && \
+ apt-get install --assume-yes --no-install-recommends \
+ python3 \
+ build-essential \
+ git \
+ python3-pip \
+ python3-wheel \
+ python3-venv \
+ python3-dev \
+ default-libmysqlclient-dev \
+ libffi-dev \
+ libjpeg-dev \
+ libmemcached-dev \
+ zlib1g-dev \
+ libssl-dev && \
+ mkdir -p /app /data /venv && \
+ chown -R nobody:nogroup /app /data /venv
+
+WORKDIR /app
+USER nobody:nogroup
+COPY requirements.txt /data/
+# Install Python deps
+RUN python3 -m venv --copies /venv && \
+ /venv/bin/pip install -U pip setuptools && \
+ /venv/bin/pip install -r /data/requirements.txt
+
+
+# App container
+FROM debian:stretch-slim AS app
+
+ENV LANG=C.UTF-8 \
+ DEBIAN_FRONTEND=noninteractive \
+ # Extra python env
+ PYTHONDONTWRITEBYTECODE=1 \
+ PYTHONUNBUFFERED=1 \
+ # Path to local settings
+ LOCAL_SETTINGS_PY=/app/docker/local_settings.docker.py
+
+# do all of this in one RUN to limit final image size
+RUN apt-get update && \
+ apt-get install --assume-yes --no-install-recommends \
+ ca-certificates \
+ intltool \
+ libjpeg62-turbo \
+ libmariadbclient18 \
+ libmemcached11 \
+ libssl1.1 \
+ python3 \
+ sqlite3 \
+ wget \
+ zlib1g && \
+ rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+USER nobody:nogroup
+
+# copy in Python environment
+COPY --from=builder /venv /venv
+# copy helpers and configs
+COPY ./docker/entrypoint.sh ./docker/irc_userlist /data/
+
+EXPOSE 8000
+CMD ["/bin/bash", "/data/entrypoint.sh"]
Modified: docker/entrypoint.sh
70 lines changed, 70 insertions(+), 0 deletions(-)
===================================================================
@@ -0,0 +1,70 @@
+#!/bin/bash
+#
+# LICENCE: This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+set -e
+
+mkdir -p /app/docker/data
+
+# setup database if it does not exist
+if [ ! -f "/app/docker/data/geany_dev.db" ]; then
+ echo "==== Setup database ===="
+ /venv/bin/python manage.py reset_db --noinput
+ /venv/bin/python manage.py migrate --run-syncdb --noinput
+ echo 'DELETE FROM auth_permission;
+ DELETE FROM django_content_type;
+ DELETE FROM django_site;' | /venv/bin/python manage.py dbshell
+ /venv/bin/python manage.py loaddata database.json
+fi
+if [ ! -f "/app/docker/data/geany_dev_nightlybuilds.db" ]; then
+ /venv/bin/python manage.py migrate --database nightlybuilds --run-syncdb --noinput
+ /venv/bin/python manage.py loaddata --database nightlybuilds database_nightlybuilds.json
+fi
+
+# screenshots
+echo "==== Download screenshots ===="
+SCREENSHOTS="$(sqlite3 docker/data/geany_dev.db 'select file from galleries_galleryimage;')"
+# add homepage screenshots
+SCREENSHOTS="${SCREENSHOTS}
+uploads/screenshots/homepage/geany_dark_2019-05-20.png
+uploads/screenshots/homepage/geany_light_php-2019-06-15.png
+uploads/screenshots/homepage/geany_windows_classic-2019-06-09.png
+"
+pushd /app/docker/data
+for screenshot in ${SCREENSHOTS}; do
+ if [ ! -f "/app/docker/data/${screenshot}" ]; then
+ mkdir -p $(dirname "/app/docker/data/${screenshot}")
+ wget \
+ --no-verbose \
+ --output-document="/app/docker/data/${screenshot}" \
+ "https://www.geany.org/media/${screenshot}"
+ fi
+done
+popd
+
+# generate i18n stats if not already available
+if [ ! -d "/app/docker/data/i18n" ]; then
+ echo "==== Update I18N statistics ===="
+ wget --no-verbose https://download.geany.org/geany_git.tar.gz -O /tmp/geany_git.tar.gz
+ /venv/bin/python manage.py generate_i18n_statistics
+ rm -f /tmp/geany_git.tar.gz
+fi
+
+# sync page contents from GIT back to the database
+echo "==== Sync page contents ===="
+/venv/bin/python /app/manage.py sync_pages
+
+# start the server
+/venv/bin/python /app/manage.py runserver 0.0.0.0:8000
Modified: docker/irc_userlist
5 lines changed, 5 insertions(+), 0 deletions(-)
===================================================================
@@ -0,0 +1,5 @@
+dummy-user-1
+dummy-user-2
+dummy-user-3
+dummy-user-4
+dummy-user-5
Modified: docker/local_settings.docker.py
124 lines changed, 124 insertions(+), 0 deletions(-)
===================================================================
@@ -0,0 +1,124 @@
+# coding.: utf-8
+
+# This file is exec'd from settings.py, so it has access to and can modify all
+# the variables in settings.py.
+
+# If this file is changed in development, the development server will have to
+# be manually restarted because changes will not be noticed immediately.
+
+
+DEBUG = True
+
+
+INTERNAL_IPS = ("127.0.0.1",)
+ALLOWED_HOSTS = ('127.0.0.1', 'localhost')
+
+STATIC_ROOT = '/app/static'
+MEDIA_ROOT = '/app/docker/data'
+
+
+DATABASES = {
+ 'default': {
+ 'ENGINE': 'django.db.backends.sqlite3',
+ 'NAME': os.path.join(MEDIA_ROOT, 'geany_dev.db'),
+ },
+ 'nightlybuilds': {
+ 'ENGINE': 'django.db.backends.sqlite3',
+ 'NAME': os.path.join(MEDIA_ROOT, 'geany_dev_nightlybuilds.db')
+ }
+}
+
+SECRET_KEY = "f94b410d-0401-4f25-a69e-d5cbbf7717f3-1af070a9-ebef-4efe-a779-a9706bbdfd4a"
+NEVERCACHE_KEY = "c8eb09b7-bd7e-4375-8293-6ffe53bf92e1-884d109e-6da4-4b99-bc56-075d8a498d4c"
+
+CSRF_COOKIE_HTTPONLY = True
+CSRF_COOKIE_SECURE = False
+SSL_FORCE_URL_PREFIXES = ()
+SESSION_COOKIE_SECURE = False
+USE_X_FORWARDED_HOST = False
+
+CACHES = {
+ 'default': {
+ 'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
+ }
+}
+
+FORCE_SCRIPT_NAME = ''
+SERVER_EMAIL = 'root at localhost'
+DEFAULT_FROM_EMAIL = 'root at localhost'
+
+NIGHTLYBUILDS_BASE_DIR = os.path.join(MEDIA_ROOT, 'nightly_mirror')
+IRC_USER_LIST_FILE = '/data/irc_userlist'
+
+STATIC_DOCS_GEANY_SOURCE_TARBALL = '/tmp/geany_git.tar.gz'
+STATIC_DOCS_GEANY_DESTINATION_DIR = os.path.join(MEDIA_ROOT, 'i18n')
+STATIC_DOCS_GEANY_DESTINATION_URL = os.path.join(MEDIA_URL, 'i18n')
+STATIC_DOCS_GEANY_I18N_STATISTICS_FILENAME = 'i18n_statistics.json'
+
+LOGGING = {
+ 'version': 1,
+ 'disable_existing_loggers': True,
+ 'formatters': {
+ 'verbose': {
+ 'format': '%(asctime)s %(name)s %(process)d %(threadName)s %(levelname)s %(message)s'
+ },
+ },
+ 'filters': {
+ 'require_debug_false': {
+ '()': 'django.utils.log.RequireDebugFalse'
+ },
+ },
+ 'handlers': {
+ 'console':{
+ 'level':'DEBUG',
+ 'class':'logging.StreamHandler',
+ 'formatter': 'verbose'
+ },
+ },
+ 'loggers': {
+ '': {
+ 'handlers':['console'],
+ 'level':'DEBUG',
+ 'propagate': False,
+ },
+ 'root': {
+ 'handlers':['console'],
+ 'level':'DEBUG',
+ 'propagate': False,
+ },
+ 'py.warnings': {
+ 'handlers':['console'],
+ 'propagate': True,
+ 'level':'DEBUG',
+ },
+ 'django': {
+ 'handlers':['console'],
+ 'propagate': True,
+ 'level':'DEBUG',
+ },
+ 'django.db': {
+ 'handlers':['console'],
+ 'level': 'INFO',
+ 'propagate': True,
+ },
+ 'django.template': {
+ 'handlers':['console'],
+ 'level': 'INFO',
+ 'propagate': True,
+ },
+ 'django.utils.autoreload': {
+ 'handlers': [],
+ 'level': 'INFO',
+ 'propagate': True,
+ },
+ 'MARKDOWN': {
+ 'level': 'INFO',
+ 'propagate': True,
+ },
+ 'PIL': {
+ 'handlers': [],
+ 'level': 'INFO',
+ 'propagate': True,
+ },
+ }
+}
Modified: geany/settings.py
4 lines changed, 3 insertions(+), 1 deletions(-)
===================================================================
@@ -594,7 +594,9 @@
# Instead of doing "from .local_settings import *", we use exec so that
# local_settings has full access to everything defined in this module.
-filename = os.path.join(PROJECT_APP_PATH, 'local_settings.py') # pylint: disable=invalid-name
+# pylint: disable=invalid-name
+local_settings_file_name = os.environ.get('LOCAL_SETTINGS_PY', 'local_settings.py')
+filename = os.path.join(PROJECT_APP_PATH, local_settings_file_name) # pylint: disable=invalid-name
if os.path.exists(filename):
import sys
from importlib.util import module_from_spec, spec_from_file_location
Modified: tox.ini
8 lines changed, 4 insertions(+), 4 deletions(-)
===================================================================
@@ -32,7 +32,7 @@ commands =
{envbindir}/pylint --rcfile=tox.ini {[tox]geany_modules}
[flake8]
-exclude = build,.git,docs,migrations,local_settings.py
+exclude = build,.git,docs,migrations,local_settings.py,local_settings.docker.py
ignore = E127,E128,
max-line-length = 100
@@ -46,12 +46,12 @@ sections = FUTURE,STDLIB,THIRDPARTY,FIRSTPARTY,LOCALFOLDER
lines_after_imports = 2
from_first = true
include_trailing_comma = true
-skip = local_settings.py
+skip = local_settings.py,local_settings.docker.py
-# the following secions are for pylint
+# the following sections are for pylint
[MASTER]
ignore=.git
-ignore-patterns=local_settings.py
+ignore-patterns=local_settings.py,local_settings.docker.py
persistent=no
load-plugins=pylint_django
--------------
This E-Mail was brought to you by github_commit_mail.py (Source: https://github.com/geany/infrastructure).
More information about the Commits
mailing list