Galaxy improvements (#628)

* galaxy: improve logging and performance reporting

* galaxy: add a full galaxy state test

* galaxy: optimize user self score computation

* galaxy: add 'generate_galaxy_test_data' command for development at scale

* galaxy: big refactor

Main changes:
  - Multiple Galaxy objects can now exist at the same time in DB. This allows for ruling a new galaxy while still
    displaying the old one.
  - The criteria to quickly know whether a user is a possible citizen is now a simple query on picture count. This
    avoids a very complicated query to database, that could often result in huge working memory load. With this change,
    it should be possible to run the galaxy even on a vanilla Postgres that didn't receive fine tuning for the Sith's
    galaxy.

* galaxy: template: make the galaxy graph work and be usable with a lot of stars

- Display focused star and its connections clearly
- Display star label faintly by default for other stars to avoid overloading the graph
- Hide non-focused lanes
- Avoid clicks on non-highlighted, too far stars
- Make the canva adapt its width to initial screen size, doesn't work dynamically

* galaxy: better docstrings

* galaxy: use bulk_create whenever possible

This is a big performance gain, especially for the tests.

Examples:

----

`./manage.py test galaxy.tests.GalaxyTest.test_full_galaxy_state`

Measurements averaged over 3 run on *my machine*™:
Before: 2min15s
After: 1m41s

----

`./manage.py generate_galaxy_test_data --user-pack-count 1`

Before: 48s
After: 25s

----

`./manage.py rule_galaxy` (for 600 citizen, corresponding to 1 user-pack)

Before: 14m4s
After: 12m34s

* core: populate: use a less ambiguous 'timezone.now()'

When running the tests around midnight, the day is changing, leading to some values being offset to the next day
depending on the timezone, and making some tests to fail. This ensure to use a less ambiguous `now` when populating
the database.

* write more extensive documentation

- add documentation to previously documented classes and functions and refactor some of the documented one, in accordance to the PEP257 and ReStructuredText standards ;
- add some type hints ;
- use a NamedTuple for the `Galaxy.compute_users_score` method instead of a raw tuple. Also change a little bit the logic in the function which call the latter ;
- add some additional parameter checks on a few functions ;
- change a little bit the logic of the log level setting for the galaxy related commands.

* galaxy: tests: split Model and View for more efficient data usage

---------

Co-authored-by: maréchal <thgirod@hotmail.com>

.gitignore

@@ -11,6 +11,7 @@ dist/
 env/
 doc/html
 data/
+galaxy/test_galaxy_state.json
 /static/
 sith/settings_custom.py
 sith/search_indexes/

core/management/commands/populate.py

@@ -208,6 +208,8 @@ Welcome to the wiki page!
 
         # Here we add a lot of test datas, that are not necessary for the Sith, but that provide a basic development environment
         if not options["prod"]:
+            self.now = timezone.now().replace(hour=12)
+
             # Adding user Skia
             skia = User(
                 username="skia",
@@ -914,6 +916,7 @@ Welcome to the wiki page!
             Membership(
                 user=comunity,
                 club=bar_club,
+                start_date=self.now,
                 role=settings.SITH_CLUB_ROLES_ID["Board member"],
             ).save()
             # Adding user tutu
@@ -1072,7 +1075,7 @@ Welcome to the wiki page!
             ForumTopic(forum=hall)
 
             # News
-            friday = timezone.now()
+            friday = self.now
             while friday.weekday() != 4:
                 friday += timedelta(hours=6)
             friday.replace(hour=20, minute=0, second=0)
@@ -1090,8 +1093,8 @@ Welcome to the wiki page!
             n.save()
             NewsDate(
                 news=n,
-                start_date=timezone.now() + timedelta(hours=70),
-                end_date=timezone.now() + timedelta(hours=72),
+                start_date=self.now + timedelta(hours=70),
+                end_date=self.now + timedelta(hours=72),
             ).save()
             n = News(
                 title="Repas barman",
@@ -1107,8 +1110,8 @@ Welcome to the wiki page!
             n.save()
             NewsDate(
                 news=n,
-                start_date=timezone.now() + timedelta(hours=72),
-                end_date=timezone.now() + timedelta(hours=84),
+                start_date=self.now + timedelta(hours=72),
+                end_date=self.now + timedelta(hours=84),
             ).save()
             n = News(
                 title="Repas fromager",
@@ -1123,8 +1126,8 @@ Welcome to the wiki page!
             n.save()
             NewsDate(
                 news=n,
-                start_date=timezone.now() + timedelta(hours=96),
-                end_date=timezone.now() + timedelta(hours=100),
+                start_date=self.now + timedelta(hours=96),
+                end_date=self.now + timedelta(hours=100),
             ).save()
             n = News(
                 title="SdF",
@@ -1140,7 +1143,7 @@ Welcome to the wiki page!
             NewsDate(
                 news=n,
                 start_date=friday + timedelta(hours=24 * 7 + 1),
-                end_date=timezone.now() + timedelta(hours=24 * 7 + 9),
+                end_date=self.now + timedelta(hours=24 * 7 + 9),
             ).save()
             # Weekly
             n = News(
@@ -1271,28 +1274,28 @@ Welcome to the wiki page!
                 club=troll,
                 role=9,
                 description="Padawan Troll",
-                start_date=timezone.now() - timedelta(days=17),
+                start_date=self.now - timedelta(days=17),
             ).save()
             Membership(
                 user=krophil,
                 club=troll,
                 role=10,
                 description="Maitre Troll",
-                start_date=timezone.now() - timedelta(days=200),
+                start_date=self.now - timedelta(days=200),
             ).save()
             Membership(
                 user=skia,
                 club=troll,
                 role=2,
                 description="Grand Ancien Troll",
-                start_date=timezone.now() - timedelta(days=400),
-                end_date=timezone.now() - timedelta(days=86),
+                start_date=self.now - timedelta(days=400),
+                end_date=self.now - timedelta(days=86),
             ).save()
             Membership(
                 user=richard,
                 club=troll,
                 role=2,
                 description="",
-                start_date=timezone.now() - timedelta(days=200),
-                end_date=timezone.now() - timedelta(days=100),
+                start_date=self.now - timedelta(days=200),
+                end_date=self.now - timedelta(days=100),
             ).save()

galaxy/management/commands/generate_galaxy_test_data.py

@@ -0,0 +1,411 @@
+# -*- coding:utf-8 -*
+#
+# Copyright 2023
+# - Skia <skia@hya.sk>
+#
+# Ce fichier fait partie du site de l'Association des Étudiants de l'UTBM,
+# http://ae.utbm.fr.
+#
+# This program is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License a published by the Free Software
+# Foundation; either version 3 of the License, or (at your option) any later
+# version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
+# details.
+#
+# You should have received a copy of the GNU General Public License along with
+# this program; if not, write to the Free Sofware Foundation, Inc., 59 Temple
+# Place - Suite 330, Boston, MA 02111-1307, USA.
+#
+#
+import warnings
+from typing import Final, Optional
+
+from django.conf import settings
+from django.core.files.base import ContentFile
+from django.core.management.base import BaseCommand
+from django.utils import timezone
+
+from datetime import timedelta
+
+import logging
+
+from club.models import Club, Membership
+from core.models import User, Group, Page, SithFile
+from subscription.models import Subscription
+from sas.models import Album, Picture, PeoplePictureRelation
+
+
+RED_PIXEL_PNG: Final[bytes] = (
+    b"\x89\x50\x4e\x47\x0d\x0a\x1a\x0a\x00\x00\x00\x0d\x49\x48\x44\x52"
+    b"\x00\x00\x00\x01\x00\x00\x00\x01\x08\x02\x00\x00\x00\x90\x77\x53"
+    b"\xde\x00\x00\x00\x0c\x49\x44\x41\x54\x08\xd7\x63\xf8\xcf\xc0\x00"
+    b"\x00\x03\x01\x01\x00\x18\xdd\x8d\xb0\x00\x00\x00\x00\x49\x45\x4e"
+    b"\x44\xae\x42\x60\x82"
+)
+
+USER_PACK_SIZE: Final[int] = 1000
+
+
+class Command(BaseCommand):
+    help = "Procedurally generate representative data for developing the Galaxy"
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.now = timezone.now().replace(hour=12)
+
+        self.users: Optional[list[User]] = None
+        self.clubs: Optional[list[Club]] = None
+        self.picts: Optional[list[Picture]] = None
+        self.pictures_tags: Optional[list[PeoplePictureRelation]] = None
+
+    def add_arguments(self, parser):
+        parser.add_argument(
+            "--user-pack-count",
+            help=f"Number of packs of {USER_PACK_SIZE} users to create",
+            type=int,
+            default=1,
+        )
+        parser.add_argument(
+            "--club-count", help="Number of clubs to create", type=int, default=50
+        )
+
+    def handle(self, *args, **options):
+        self.logger = logging.getLogger("main")
+        if options["verbosity"] < 0 or 2 < options["verbosity"]:
+            warnings.warn("verbosity level should be between 0 and 2 included")
+
+        if options["verbosity"] == 2:
+            self.logger.setLevel(logging.DEBUG)
+        elif options["verbosity"] == 1:
+            self.logger.setLevel(logging.INFO)
+        else:
+            self.logger.setLevel(logging.ERROR)
+
+        self.logger.info("The Galaxy is being populated by the Sith.")
+
+        self.logger.info("Cleaning old Galaxy population")
+        Club.objects.filter(unix_name__startswith="galaxy-").delete()
+        Group.objects.filter(name__startswith="galaxy-").delete()
+        Page.objects.filter(name__startswith="galaxy-").delete()
+        User.objects.filter(username__startswith="galaxy-").delete()
+        Picture.objects.filter(name__startswith="galaxy-").delete()
+        Album.objects.filter(name__startswith="galaxy-").delete()
+        self.logger.info("Done. Populating a new Galaxy")
+
+        self.NB_USERS = options["user_pack_count"] * USER_PACK_SIZE
+        self.NB_CLUBS = options["club_count"]
+
+        root = User.objects.filter(username="root").first()
+        sas = SithFile.objects.get(id=settings.SITH_SAS_ROOT_DIR_ID)
+        self.galaxy_album = Album.objects.create(
+            name="galaxy-register-file",
+            owner=root,
+            is_moderated=True,
+            is_in_sas=True,
+            parent=sas,
+        )
+
+        self.make_clubs()
+        self.make_users()
+        self.make_families()
+        self.make_club_memberships()
+        self.make_pictures()
+        self.make_pictures_memberships()
+        half_pack = USER_PACK_SIZE // 2
+        for u in range(half_pack, self.NB_USERS, half_pack):
+            self.make_important_citizen(u)
+
+    def make_clubs(self):
+        """
+        Create all the clubs (:class:`club.models.Club`)
+        and store them in `self.clubs` for fast access later.
+        Don't create the meta groups (:class:`core.models.MetaGroup`)
+        nor the pages of the clubs (:class:`core.models.Page`)
+        """
+        self.clubs = []
+        for i in range(self.NB_CLUBS):
+            self.clubs.append(Club(unix_name=f"galaxy-club-{i}", name=f"club-{i}"))
+        # We don't need to create corresponding groups here, as the Galaxy doesn't care about them
+        Club.objects.bulk_create(self.clubs)
+        self.clubs = Club.objects.filter(unix_name__startswith="galaxy-").all()
+
+    def make_users(self):
+        """
+        Create all the users and store them in `self.users` for fast access later.
+
+        Also create a subscription for all the generated users.
+        """
+        self.users = []
+        for i in range(self.NB_USERS):
+            u = User(
+                username=f"galaxy-user-{i}",
+                email=f"{i}@galaxy.test",
+                first_name="Citizen",
+                last_name=f"n°{i}",
+            )
+            self.logger.info(f"Creating {u}")
+            self.users.append(u)
+        User.objects.bulk_create(self.users)
+        self.users = User.objects.filter(username__startswith="galaxy-").all()
+
+        # now that users are created, create their subscription
+        subs = []
+        for i in range(self.NB_USERS):
+            u = self.users[i]
+            self.logger.info(f"Registering {u}")
+            subs.append(
+                Subscription(
+                    member=u,
+                    subscription_start=Subscription.compute_start(
+                        self.now - timedelta(days=self.NB_USERS - i)
+                    ),
+                    subscription_end=Subscription.compute_end(duration=2),
+                )
+            )
+        Subscription.objects.bulk_create(subs)
+
+    def make_families(self):
+        """
+        Generate the godfather/godchild relations for the users contained in :attr:`self.users`.
+
+        The :meth:`make_users` method must have been called beforehand.
+
+        This will iterate on all citizen after the 200th.
+        Then it will take 14 other citizen among the previous 200
+        (godfathers are usually older), and apply another
+        heuristic to determine whether they should have a family link
+        """
+        if self.users is None:
+            raise RuntimeError(
+                "The `make_users()` method must be called before `make_families()`"
+            )
+        for i in range(200, self.NB_USERS):
+            godfathers = []
+            for j in range(i - 200, i, 14):  # this will loop 14 times (14² = 196)
+                if (i / 10) % 10 == (i + j) % 10:
+                    u1 = self.users[i]
+                    u2 = self.users[j]
+                    self.logger.info(f"Making {u2} the godfather of {u1}")
+                    godfathers.append(u2)
+                u1.godfathers.set(godfathers)
+
+    def make_club_memberships(self):
+        """
+        Assign users to clubs and give them a role in a pseudo-random way.
+
+        The :meth:`make_users` and :meth:`make_clubs` methods
+        must have been called beforehand.
+
+        Work by making multiples passes on all users to affect
+        them some pseudo-random roles in some clubs.
+        The multiple passes are useful to get some variations over who goes where.
+        Each pass for each user has a chance to affect her to two different clubs,
+        increasing a bit more the created chaos, while remaining purely deterministic.
+        """
+        if self.users is None:
+            raise RuntimeError(
+                "The `make_users()` method must be called before `make_club_memberships()`"
+            )
+        if self.clubs is None:
+            raise RuntimeError(
+                "The `make_clubs()` method must be called before `make_club_memberships()`"
+            )
+        memberships = []
+        for i in range(1, 11):  # users can be in up to 20 clubs
+            self.logger.info(f"Club membership, pass {i}")
+            for uid in range(
+                i, self.NB_USERS, i
+            ):  # Pass #1 will make sure every user is at least in one club
+                user = self.users[uid]
+                club = self.clubs[(uid + i**2) % self.NB_CLUBS]
+
+                start = self.now - timedelta(
+                    days=(((self.NB_USERS - uid) * i) // 110)
+                )  # older users were in clubs before newer users
+                end = start + timedelta(days=180)  # about one semester
+                self.logger.debug(
+                    f"Making {user} a member of club {club} from {start} to {end}"
+                )
+                memberships.append(
+                    Membership(
+                        user=user,
+                        club=club,
+                        role=(uid + i) % 10 + 1,  # spread the different roles
+                        start_date=start,
+                        end_date=end,
+                    )
+                )
+
+            for uid in range(
+                10 + i * 2, self.NB_USERS, 10 + i * 2
+            ):  # Make a second affectation that will skip most users, to make a few citizen more important
+                user = self.users[uid]
+                club = self.clubs[(uid + i**3) % self.NB_CLUBS]
+
+                start = self.now - timedelta(
+                    days=(((self.NB_USERS - uid) * i) // 100)
+                )  # older users were in clubs before newer users
+                end = start + timedelta(days=180)  # about one semester
+                self.logger.debug(
+                    f"Making {user} a member of club {club} from {start} to {end}"
+                )
+                memberships.append(
+                    Membership(
+                        user=user,
+                        club=club,
+                        role=((uid // 10) + i) % 10 + 1,  # spread the different roles
+                        start_date=start,
+                        end_date=end,
+                    )
+                )
+        Membership.objects.bulk_create(memberships)
+
+    def make_pictures(self):
+        """
+        Create pictures for users to be tagged on later.
+
+        The :meth:`make_users` method must have been called beforehand.
+        """
+        if self.users is None:
+            raise RuntimeError(
+                "The `make_users()` method must be called before `make_families()`"
+            )
+        self.picts = []
+        # Create twice as many pictures as users
+        for i in range(self.NB_USERS * 2):
+            u = self.users[i % self.NB_USERS]
+            self.logger.info(f"Making Picture {i // self.NB_USERS} for {u}")
+            self.picts.append(
+                Picture(
+                    owner=u,
+                    name=f"galaxy-picture {u} {i // self.NB_USERS}",
+                    is_moderated=True,
+                    is_folder=False,
+                    parent=self.galaxy_album,
+                    is_in_sas=True,
+                    file=ContentFile(RED_PIXEL_PNG),
+                    compressed=ContentFile(RED_PIXEL_PNG),
+                    thumbnail=ContentFile(RED_PIXEL_PNG),
+                    mime_type="image/png",
+                    size=len(RED_PIXEL_PNG),
+                )
+            )
+            self.picts[i].file.name = self.picts[i].name
+            self.picts[i].compressed.name = self.picts[i].name
+            self.picts[i].thumbnail.name = self.picts[i].name
+        Picture.objects.bulk_create(self.picts)
+        self.picts = Picture.objects.filter(name__startswith="galaxy-").all()
+
+    def make_pictures_memberships(self):
+        """
+        Assign users to pictures and make enough of them for our
+        created users to be eligible for promotion as citizen.
+
+        See :meth:`galaxy.models.Galaxy.rule` for details on promotion to citizen.
+        """
+        self.pictures_tags = []
+
+        # We don't want to handle limits, users in the middle will be far enough
+        def _tag_neighbors(uid, neighbor_dist, pict_offset, pict_dist):
+            u2 = self.users[uid - neighbor_dist]
+            u3 = self.users[uid + neighbor_dist]
+            self.pictures_tags += [
+                PeoplePictureRelation(user=u2, picture=self.picts[uid + pict_offset]),
+                PeoplePictureRelation(user=u3, picture=self.picts[uid + pict_offset]),
+                PeoplePictureRelation(user=u2, picture=self.picts[uid - pict_dist]),
+                PeoplePictureRelation(user=u3, picture=self.picts[uid - pict_dist]),
+                PeoplePictureRelation(user=u2, picture=self.picts[uid + pict_dist]),
+                PeoplePictureRelation(user=u3, picture=self.picts[uid + pict_dist]),
+            ]
+
+        for uid in range(200, self.NB_USERS - 200):
+            u1 = self.users[uid]
+            self.logger.info(f"Pictures of {u1}")
+            self.pictures_tags += [
+                PeoplePictureRelation(user=u1, picture=self.picts[uid]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid - 14]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid + 14]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid - 20]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid + 20]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid - 21]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid + 21]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid - 22]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid + 22]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid - 30]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid + 30]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid - 31]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid + 31]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid - 32]),
+                PeoplePictureRelation(user=u1, picture=self.picts[uid + 32]),
+            ]
+
+            if uid % 3 == 0:
+                _tag_neighbors(uid, 1, 0, 40)
+            if uid % 5 == 0:
+                _tag_neighbors(uid, 2, 0, 50)
+            if uid % 10 == 0:
+                _tag_neighbors(uid, 3, 0, 60)
+            if uid % 20 == 0:
+                _tag_neighbors(uid, 5, 0, 70)
+            if uid % 25 == 0:
+                _tag_neighbors(uid, 10, 0, 80)
+
+            if uid % 2 == 1:
+                _tag_neighbors(uid, 1, self.NB_USERS, 90)
+            if uid % 15 == 0:
+                _tag_neighbors(uid, 5, self.NB_USERS, 100)
+            if uid % 30 == 0:
+                _tag_neighbors(uid, 4, self.NB_USERS, 110)
+        PeoplePictureRelation.objects.bulk_create(self.pictures_tags)
+
+    def make_important_citizen(self, uid: int):
+        """
+        Make the user whose uid is given in parameter a more important citizen,
+        thus triggering many more connections to others (lanes)
+        and dragging him towards the center of the Galaxy.
+
+        This promotion is obtained by adding more family links
+        and by tagging the user in more pictures.
+
+        The users chosen to be added to this user's family shall
+        also be tagged in more pictures, thus making them also
+        more important.
+
+        :param uid: the id of the user to make more important
+        """
+        u1 = self.users[uid]
+        u2 = self.users[uid - 100]
+        u3 = self.users[uid + 100]
+        u1.godfathers.add(u2)
+        u1.godchildren.add(u3)
+        self.logger.info(f"{u1} will be important and close to {u2} and {u3}")
+        pictures_tags = []
+        for p in range(  # Mix them with other citizen for more chaos
+            uid - 400, uid - 200
+        ):
+            # users may already be on the pictures
+            if not self.picts[p].people.filter(user=u1).exists():
+                pictures_tags.append(
+                    PeoplePictureRelation(user=u1, picture=self.picts[p])
+                )
+            if not self.picts[p].people.filter(user=u2).exists():
+                pictures_tags.append(
+                    PeoplePictureRelation(user=u2, picture=self.picts[p])
+                )
+            if not self.picts[p + self.NB_USERS].people.filter(user=u1).exists():
+                pictures_tags.append(
+                    PeoplePictureRelation(
+                        user=u1, picture=self.picts[p + self.NB_USERS]
+                    )
+                )
+            if not self.picts[p + self.NB_USERS].people.filter(user=u2).exists():
+                pictures_tags.append(
+                    PeoplePictureRelation(
+                        user=u2, picture=self.picts[p + self.NB_USERS]
+                    )
+                )
+        PeoplePictureRelation.objects.bulk_create(pictures_tags)

galaxy/management/commands/rule_galaxy.py

@@ -21,6 +21,7 @@
 # Place - Suite 330, Boston, MA 02111-1307, USA.
 #
 #
+import warnings
 
 from django.core.management.base import BaseCommand
 from django.db import connection
@@ -41,19 +42,21 @@ class Command(BaseCommand):
 
     def handle(self, *args, **options):
         logger = logging.getLogger("main")
-        if options["verbosity"] > 1:
+        if options["verbosity"] < 0 or 2 < options["verbosity"]:
+            warnings.warn("verbosity level should be between 0 and 2 included")
+
+        if options["verbosity"] == 2:
             logger.setLevel(logging.DEBUG)
-        elif options["verbosity"] > 0:
+        elif options["verbosity"] == 1:
             logger.setLevel(logging.INFO)
         else:
-            logger.setLevel(logging.NOTSET)
+            logger.setLevel(logging.ERROR)
 
         logger.info("The Galaxy is being ruled by the Sith.")
-        Galaxy.rule()
-        logger.info(
-            "Caching current Galaxy state for a quicker display of the Empire's power."
-        )
-        Galaxy.make_state()
+        galaxy = Galaxy.objects.create()
+        galaxy.rule()
+        logger.info("Sending old galaxies' remains to garbage.")
+        Galaxy.objects.filter(state__isnull=True).delete()
 
         logger.info("Ruled the galaxy in {} queries.".format(len(connection.queries)))
         if options["verbosity"] > 2:

galaxy/migrations/0002_auto_20230412_1130.py

@@ -0,0 +1,45 @@
+# Generated by Django 3.2.16 on 2023-04-12 09:30
+
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
+        ("galaxy", "0001_initial"),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name="galaxy",
+            options={"ordering": ["pk"]},
+        ),
+        migrations.AddField(
+            model_name="galaxystar",
+            name="galaxy",
+            field=models.ForeignKey(
+                null=True,
+                on_delete=django.db.models.deletion.CASCADE,
+                related_name="stars",
+                to="galaxy.galaxy",
+                verbose_name="the galaxy this star belongs to",
+            ),
+        ),
+        migrations.AlterField(
+            model_name="galaxy",
+            name="state",
+            field=models.JSONField(null=True, verbose_name="The galaxy current state"),
+        ),
+        migrations.AlterField(
+            model_name="galaxystar",
+            name="owner",
+            field=models.ForeignKey(
+                on_delete=django.db.models.deletion.CASCADE,
+                related_name="stars",
+                to=settings.AUTH_USER_MODEL,
+                verbose_name="star owner",
+            ),
+        ),
+    ]

galaxy/models.py

@@ -22,16 +22,19 @@
 #
 #
 
+from __future__ import annotations
+
 import math
 import logging
+import time
+
+from typing import List, TypedDict, NamedTuple, Union, Optional
 
-from typing import Tuple
 from django.db import models
 from django.db.models import Q, Case, F, Value, When, Count
 from django.db.models.functions import Concat
 from django.utils import timezone
 from django.utils.translation import gettext_lazy as _
-from typing import List, TypedDict
 
 from core.models import User
 from club.models import Club
@@ -40,30 +43,58 @@ from sas.models import Picture
 
 class GalaxyStar(models.Model):
     """
-    This class defines a star (vertex -> user) in the galaxy graph, storing a reference to its owner citizen, and being
-    referenced by GalaxyLane.
+    Define a star (vertex -> user) in the galaxy graph,
+    storing a reference to its owner citizen.
 
-    It also stores the individual mass of this star, used to push it towards the center of the galaxy.
+    Stars are linked to each others through the :class:`GalaxyLane` model.
+
+    Each GalaxyStar has a mass which push it towards the center of the galaxy.
+    This mass is proportional to the number of pictures the owner of the star
+    is tagged on.
     """
 
-    owner = models.OneToOneField(
+    owner = models.ForeignKey(
         User,
         verbose_name=_("star owner"),
-        related_name="galaxy_user",
+        related_name="stars",
         on_delete=models.CASCADE,
     )
     mass = models.PositiveIntegerField(
         _("star mass"),
         default=0,
     )
+    galaxy = models.ForeignKey(
+        "Galaxy",
+        verbose_name=_("the galaxy this star belongs to"),
+        related_name="stars",
+        on_delete=models.CASCADE,
+        null=True,
+    )
 
     def __str__(self):
         return str(self.owner)
 
 
+@property
+def current_star(self) -> Optional[GalaxyStar]:
+    """
+    The star of this user in the :class:`Galaxy`.
+    Only take into account the most recent active galaxy.
+
+    :return: The star of this user if there is an active Galaxy
+             and this user is a citizen of it, else ``None``
+    """
+    return self.stars.filter(galaxy=Galaxy.get_current_galaxy()).last()
+
+
+# Adding a shortcut to User class for getting its star belonging to the latest ruled Galaxy
+setattr(User, "current_star", current_star)
+
+
 class GalaxyLane(models.Model):
     """
-    This class defines a lane (edge -> link between galaxy citizen) in the galaxy map, storing a reference to both its
+    Define a lane (edge -> link between galaxy citizen)
+    in the galaxy map, storing a reference to both its
     ends and the distance it covers.
     Score details between citizen owning the stars is also stored here.
     """
@@ -110,7 +141,35 @@ class GalaxyDict(TypedDict):
     links: List
 
 
+class RelationScore(NamedTuple):
+    family: int
+    pictures: int
+    clubs: int
+
+
 class Galaxy(models.Model):
+    """
+    The Galaxy, a graph linking the active users between each others.
+    The distance between two users is given by a relation score which takes
+    into account a few parameter like the number of pictures they are both tagged on,
+    the time during which they were in the same clubs and whether they are
+    in the same family.
+
+    The citizens of the Galaxy are represented by :class:`GalaxyStar`
+    and their relations by :class:`GalaxyLane`.
+
+    Several galaxies can coexist. In this case, only the most recent active one
+    shall usually be taken into account.
+    This is useful to keep the current galaxy while generating a new one
+    and swapping them only at the very end.
+
+    Please take into account that generating the galaxy is a very expensive
+    operation. For this reason, try not to call the :meth:`rule` method more
+    than once a day in production.
+
+    To quickly access to the state of a galaxy, use the :attr:`state` attribute.
+    """
+
     logger = logging.getLogger("main")
 
     GALAXY_SCALE_FACTOR = 2_000
@@ -118,77 +177,44 @@ class Galaxy(models.Model):
     PICTURE_POINTS = 2  # Equivalent to two days as random members of a club.
     CLUBS_POINTS = 1  # One day together as random members in a club is one point.
 
-    state = models.JSONField("current state")
+    state = models.JSONField(_("The galaxy current state"), null=True)
 
-    @staticmethod
-    def make_state() -> None:
-        """
-        Compute JSON structure to send to 3d-force-graph: https://github.com/vasturiano/3d-force-graph/
-        """
-        without_nickname = Concat(
-            F("owner__first_name"), Value(" "), F("owner__last_name")
-        )
-        with_nickname = Concat(
-            F("owner__first_name"),
-            Value(" "),
-            F("owner__last_name"),
-            Value(" ("),
-            F("owner__nick_name"),
-            Value(")"),
-        )
-        stars = GalaxyStar.objects.annotate(
-            owner_name=Case(
-                When(owner__nick_name=None, then=without_nickname),
-                default=with_nickname,
-            )
-        )
-        lanes = GalaxyLane.objects.annotate(
-            star1_owner=F("star1__owner__id"),
-            star2_owner=F("star2__owner__id"),
-        )
-        json = GalaxyDict(
-            nodes=[
-                StarDict(id=star.owner_id, name=star.owner_name, mass=star.mass)
-                for star in stars
-            ],
-            links=[],
-        )
-        # Make bidirectional links
-        # TODO: see if this impacts performance with a big graph
-        for path in lanes:
-            json["links"].append(
-                {
-                    "source": path.star1_owner,
-                    "target": path.star2_owner,
-                    "value": path.distance,
-                }
-            )
-            json["links"].append(
-                {
-                    "source": path.star2_owner,
-                    "target": path.star1_owner,
-                    "value": path.distance,
-                }
-            )
-        Galaxy.objects.all().delete()
-        Galaxy(state=json).save()
+    class Meta:
+        ordering = ["pk"]
+
+    def __str__(self):
+        stars_count = self.stars.count()
+        s = f"GLX-ID{self.pk}-SC{stars_count}-"
+        if self.state is None:
+            s += "CHS"  # CHAOS
+        else:
+            s += "RLD"  # RULED
+        return s
+
+    @classmethod
+    def get_current_galaxy(
+        cls,
+    ) -> Galaxy:  # __future__.annotations is required for this
+        return Galaxy.objects.filter(state__isnull=False).last()
 
     ###################
     # User self score #
     ###################
 
     @classmethod
-    def compute_user_score(cls, user) -> int:
+    def compute_user_score(cls, user: User) -> int:
         """
-        This compute an individual score for each citizen. It will later be used by the graph algorithm to push
+        Compute an individual score for each citizen.
+        It will later be used by the graph algorithm to push
         higher scores towards the center of the galaxy.
 
         Idea: This could be added to the computation:
-              - Forum posts
-              - Picture count
-              - Counter consumption
-              - Barman time
-              - ...
+
+        - Forum posts
+        - Picture count
+        - Counter consumption
+        - Barman time
+        - ...
         """
         user_score = 1
         user_score += cls.query_user_score(user)
@@ -203,7 +229,11 @@ class Galaxy(models.Model):
         return user_score
 
     @classmethod
-    def query_user_score(cls, user) -> int:
+    def query_user_score(cls, user: User) -> int:
+        """
+        Perform the db query to get the  individual score
+        of the given user in the galaxy.
+        """
         score_query = (
             User.objects.filter(id=user.id)
             .annotate(
@@ -230,26 +260,48 @@ class Galaxy(models.Model):
     ####################
 
     @classmethod
-    def compute_users_score(cls, user1, user2) -> Tuple[int, int, int, int]:
+    def compute_users_score(cls, user1: User, user2: User) -> RelationScore:
+        """
+        Compute the relationship scores of the two given users
+        in the following fields :
+
+        - family: if they have some godfather/godchild relation
+        - pictures: in how many pictures are both tagged
+        - clubs: during how many days they were members of the same clubs
+        """
         family = cls.compute_users_family_score(user1, user2)
         pictures = cls.compute_users_pictures_score(user1, user2)
         clubs = cls.compute_users_clubs_score(user1, user2)
-        score = family + pictures + clubs
-        return score, family, pictures, clubs
+        return RelationScore(family=family, pictures=pictures, clubs=clubs)
 
     @classmethod
-    def compute_users_family_score(cls, user1, user2) -> int:
+    def compute_users_family_score(cls, user1: User, user2: User) -> int:
+        """
+        Compute the family score of the relation between the given users.
+        This takes into account mutual godfathers.
+
+        :return: 366 if user1 is the godfather of user2 (or vice versa) else 0
+        """
         link_count = User.objects.filter(
             Q(id=user1.id, godfathers=user2) | Q(id=user2.id, godfathers=user1)
         ).count()
-        if link_count:
+        if link_count > 0:
             cls.logger.debug(
                 f"\t\t- '{user1}' and '{user2}' have {link_count} direct family link"
             )
         return link_count * cls.FAMILY_LINK_POINTS
 
     @classmethod
-    def compute_users_pictures_score(cls, user1, user2) -> int:
+    def compute_users_pictures_score(cls, user1: User, user2: User) -> int:
+        """
+        Compute the pictures score of the relation between the given users.
+
+        The pictures score is obtained by counting the number
+        of :class:`Picture` in which they have been both identified.
+        This score is then multiplied by 2.
+
+        :return: The number of pictures both users have in common, times 2
+        """
         picture_count = (
             Picture.objects.filter(people__user__in=(user1,))
             .filter(people__user__in=(user2,))
@@ -262,7 +314,21 @@ class Galaxy(models.Model):
         return picture_count * cls.PICTURE_POINTS
 
     @classmethod
-    def compute_users_clubs_score(cls, user1, user2) -> int:
+    def compute_users_clubs_score(cls, user1: User, user2: User) -> int:
+        """
+        Compute the clubs score of the relation between the given users.
+
+        The club score is obtained by counting the number of days
+        during which the memberships (see :class:`club.models.Membership`)
+        of both users overlapped.
+
+        For example, if user1 was a member of Unitec from 01/01/2020 to 31/12/2021
+        (two years) and user2 was a member of the same club from 01/01/2021 to
+        31/12/2022 (also two years, but with an offset of one year), then their
+        club score is 365.
+
+        :return: the number of days during which both users were in the same club
+        """
         common_clubs = Club.objects.filter(members__in=user1.memberships.all()).filter(
             members__in=user2.memberships.all()
         )
@@ -272,6 +338,7 @@ class Galaxy(models.Model):
         score = 0
         for user1_membership in user1_memberships:
             if user1_membership.end_date is None:
+                # user1_membership.save() is not called in this function, hence this is safe
                 user1_membership.end_date = timezone.now().date()
             query = Q(  # start2 <= start1 <= end2
                 start_date__lte=user1_membership.start_date,
@@ -312,54 +379,17 @@ class Galaxy(models.Model):
     ###################
 
     @classmethod
-    def rule(cls) -> None:
-        GalaxyStar.objects.all().delete()
-        # The following is a no-op thanks to cascading, but in case that changes in the future, better keep it anyway.
-        GalaxyLane.objects.all().delete()
-        rulable_users = (
-            User.objects.filter(subscriptions__isnull=False)
-            .filter(
-                Q(godchildren__isnull=False)
-                | Q(godfathers__isnull=False)
-                | Q(pictures__isnull=False)
-                | Q(memberships__isnull=False)
-            )
-            .distinct()
-        )
-        # force fetch of the whole query to make sure there won't
-        # be any more db hits
-        # this is memory expensive but prevents a lot of db hits, therefore
-        # is far more time efficient
-        rulable_users = list(rulable_users)
-        while len(rulable_users) > 0:
-            user1 = rulable_users.pop()
-            for user2 in rulable_users:
-                cls.logger.debug("")
-                cls.logger.debug(f"\t> Ruling '{user1}' against '{user2}'")
-                star1, _ = GalaxyStar.objects.get_or_create(owner=user1)
-                star2, _ = GalaxyStar.objects.get_or_create(owner=user2)
-                if star1.mass == 0:
-                    star1.mass = cls.compute_user_score(user1)
-                    star1.save()
-                if star2.mass == 0:
-                    star2.mass = cls.compute_user_score(user2)
-                    star2.save()
-                users_score, family, pictures, clubs = cls.compute_users_score(
-                    user1, user2
-                )
-                if users_score > 0:
-                    GalaxyLane(
-                        star1=star1,
-                        star2=star2,
-                        distance=cls.scale_distance(users_score),
-                        family=family,
-                        pictures=pictures,
-                        clubs=clubs,
-                    ).save()
+    def scale_distance(cls, value: Union[int, float]) -> int:
+        """
+        Given a numeric value, return a scaled value which can
+        be used in the Galaxy's graphical interface to set the distance
+        between two stars
 
-    @classmethod
-    def scale_distance(cls, value) -> int:
+        :return: the scaled value usable in the Galaxy's 3d graph
+        """
         # TODO: this will need adjustements with the real, typical data on Taiste
+        if value == 0:
+            return 4000  # Following calculus would give us +∞, we cap it to 4000
 
         cls.logger.debug(f"\t\t> Score: {value}")
         # Invert score to draw close users together
@@ -376,3 +406,224 @@ class Galaxy(models.Model):
         )
         cls.logger.debug(f"\t\t> Scaled distance: {value}")
         return int(value)
+
+    def rule(self, picture_count_threshold=10) -> None:
+        """
+        Main function of the Galaxy.
+        Iterate over all the rulable users to promote them to citizens.
+        A citizen is a user who has a corresponding star in the Galaxy.
+        Also build up the lanes, which are the links between the different citizen.
+
+        Users who can be ruled are defined with the `picture_count_threshold`:
+        all users who are identified in a strictly lower number of pictures
+        won't be promoted to citizens.
+        This does very effectively limit the quantity of computing to do
+        and only includes users who have had a minimum of activity.
+
+        This method still remains very expensive, so think thoroughly before
+        you call it, especially in production.
+
+        :param picture_count_threshold: the minimum number of picture to have to be
+                                        included in the galaxy
+        """
+        total_time = time.time()
+        self.logger.info("Listing rulable citizen.")
+        rulable_users = (
+            User.objects.filter(subscriptions__isnull=False)
+            .annotate(pictures_count=Count("pictures"))
+            .filter(pictures_count__gt=picture_count_threshold)
+            .distinct()
+        )
+
+        # force fetch of the whole query to make sure there won't
+        # be any more db hits
+        # this is memory expensive but prevents a lot of db hits, therefore
+        # is far more time efficient
+
+        rulable_users = list(rulable_users)
+        rulable_users_count = len(rulable_users)
+        user1_count = 0
+        self.logger.info(
+            f"{rulable_users_count} citizen have been listed. Starting to rule."
+        )
+
+        stars = []
+        self.logger.info("Creating stars for all citizen")
+        for user in rulable_users:
+            star = GalaxyStar(
+                owner=user, galaxy=self, mass=self.compute_user_score(user)
+            )
+            stars.append(star)
+        GalaxyStar.objects.bulk_create(stars)
+
+        stars = {}
+        for star in GalaxyStar.objects.filter(galaxy=self):
+            stars[star.owner.id] = star
+
+        self.logger.info("Creating lanes between stars")
+        # Display current speed every $speed_count_frequency users
+        speed_count_frequency = max(rulable_users_count // 10, 1)  # ten time at most
+        global_avg_speed_accumulator = 0
+        global_avg_speed_count = 0
+        t_global_start = time.time()
+        while len(rulable_users) > 0:
+            user1 = rulable_users.pop()
+            user1_count += 1
+            rulable_users_count2 = len(rulable_users)
+
+            star1 = stars[user1.id]
+
+            user_avg_speed = 0
+            user_avg_speed_count = 0
+
+            tstart = time.time()
+            lanes = []
+            for user2_count, user2 in enumerate(rulable_users, start=1):
+                self.logger.debug("")
+                self.logger.debug(
+                    f"\t> Examining '{user1}' ({user1_count}/{rulable_users_count}) with '{user2}' ({user2_count}/{rulable_users_count2})"
+                )
+
+                star2 = stars[user2.id]
+
+                score = Galaxy.compute_users_score(user1, user2)
+                distance = self.scale_distance(sum(score))
+                if distance < 30:  # TODO: this needs tuning with real-world data
+                    lanes.append(
+                        GalaxyLane(
+                            star1=star1,
+                            star2=star2,
+                            distance=distance,
+                            family=score.family,
+                            pictures=score.pictures,
+                            clubs=score.clubs,
+                        )
+                    )
+
+                if user2_count % speed_count_frequency == 0:
+                    tend = time.time()
+                    delta = tend - tstart
+                    speed = float(speed_count_frequency) / delta
+                    user_avg_speed += speed
+                    user_avg_speed_count += 1
+                    self.logger.debug(
+                        f"\tSpeed: {speed:.2f} users per second (time for last {speed_count_frequency} citizens: {delta:.2f} second)"
+                    )
+                    tstart = time.time()
+
+            GalaxyLane.objects.bulk_create(lanes)
+
+            self.logger.info("")
+
+            t_global_end = time.time()
+            global_delta = t_global_end - t_global_start
+            speed = 1.0 / global_delta
+            global_avg_speed_accumulator += speed
+            global_avg_speed_count += 1
+            global_avg_speed = global_avg_speed_accumulator / global_avg_speed_count
+
+            self.logger.info(f" Ruling of {self} ".center(60, "#"))
+            self.logger.info(
+                f"Progression: {user1_count}/{rulable_users_count} citizen -- {rulable_users_count - user1_count} remaining"
+            )
+            self.logger.info(f"Speed: {60.0*global_avg_speed:.2f} citizen per minute")
+
+            # We can divide the computed ETA by 2 because each loop, there is one citizen less to check, and maths tell
+            # us that this averages to a division by two
+            eta = rulable_users_count2 / global_avg_speed / 2
+            eta_hours = int(eta // 3600)
+            eta_minutes = int(eta // 60 % 60)
+            self.logger.info(
+                f"ETA: {eta_hours} hours {eta_minutes} minutes ({eta / 3600 / 24:.2f} days)"
+            )
+            self.logger.info("#" * 60)
+            t_global_start = time.time()
+
+        # Here, we get the IDs of the old galaxies that we'll need to delete. In normal operation, only one galaxy
+        # should be returned, and we can't delete it yet, as it's the one still displayed by the Sith.
+        old_galaxies_pks = list(
+            Galaxy.objects.filter(state__isnull=False).values_list("pk", flat=True)
+        )
+        self.logger.info(
+            f"These old galaxies will be deleted once the new one is ready: {old_galaxies_pks}"
+        )
+
+        # Making the state sets this new galaxy as being ready. From now on, the Sith will show us to the world.
+        self.make_state()
+
+        # Avoid accident if there is nothing to delete
+        if len(old_galaxies_pks) > 0:
+            # Former galaxies can now be deleted.
+            Galaxy.objects.filter(pk__in=old_galaxies_pks).delete()
+
+        total_time = time.time() - total_time
+        total_time_hours = int(total_time // 3600)
+        total_time_minutes = int(total_time // 60 % 60)
+        total_time_seconds = int(total_time % 60)
+        self.logger.info(
+            f"{self} ruled in {total_time:.2f} seconds ({total_time_hours} hours, {total_time_minutes} minutes, {total_time_seconds} seconds)"
+        )
+
+    def make_state(self) -> None:
+        """
+        Compute JSON structure to send to 3d-force-graph: https://github.com/vasturiano/3d-force-graph/
+        """
+        self.logger.info(
+            "Caching current Galaxy state for a quicker display of the Empire's power."
+        )
+
+        without_nickname = Concat(
+            F("owner__first_name"), Value(" "), F("owner__last_name")
+        )
+        with_nickname = Concat(
+            F("owner__first_name"),
+            Value(" "),
+            F("owner__last_name"),
+            Value(" ("),
+            F("owner__nick_name"),
+            Value(")"),
+        )
+        stars = (
+            GalaxyStar.objects.filter(galaxy=self)
+            .order_by(
+                "owner"
+            )  # This helps determinism for the tests and doesn't cost much
+            .annotate(
+                owner_name=Case(
+                    When(owner__nick_name=None, then=without_nickname),
+                    default=with_nickname,
+                )
+            )
+        )
+        lanes = (
+            GalaxyLane.objects.filter(star1__galaxy=self)
+            .order_by(
+                "star1"
+            )  # This helps determinism for the tests and doesn't cost much
+            .annotate(
+                star1_owner=F("star1__owner__id"),
+                star2_owner=F("star2__owner__id"),
+            )
+        )
+        json = GalaxyDict(
+            nodes=[
+                StarDict(
+                    id=star.owner_id,
+                    name=star.owner_name,
+                    mass=star.mass,
+                )
+                for star in stars
+            ],
+            links=[],
+        )
+        for path in lanes:
+            json["links"].append(
+                {
+                    "source": path.star1_owner,
+                    "target": path.star2_owner,
+                    "value": path.distance,
+                }
+            )
+        self.state = json
+        self.save()
+        self.logger.info(f"{self} is now ready!")

galaxy/templates/galaxy/user.jinja

@@ -1,37 +1,42 @@
 {% extends "core/base.jinja" %}
 
 {% block title %}
-{% trans user_name=user.get_display_name() %}{{ user_name }}'s Galaxy{% endtrans %}
+{% trans user_name=object.get_display_name() %}{{ user_name }}'s Galaxy{% endtrans %}
 {% endblock %}
 
 {% block content %}
-{% if object.galaxy_user %}
-    <p><a onclick="focus_node(get_node_from_id({{ object.id }}))">Reset on {{ object.get_display_name() }}</a></p>
-    <p>Self score: {{ object.galaxy_user.mass }}</p>
-    <table style="width: initial;">
-        <tr>
-          <th></th>
-          <th>Citizen</th>
-          <th>Score</th>
-          <th>Distance</th>
-          <th>Family</th>
-          <th>Pictures</th>
-          <th>Clubs</th>
-        </tr>
-    {% for lane in lanes %}
-        <tr>
-          <td><a onclick="focus_node(get_node_from_id({{ lane.other_star_id }}))">Locate</a></td>
-          <td><a href="{{ url("galaxy:user", user_id=lane.other_star_id) }}">{{ lane.other_star_name }}</a></td>
-          <td>{{ lane.other_star_mass }}</td>
-          <td>{{ lane.distance }}</td>
-          <td>{{ lane.family }}</td>
-          <td>{{ lane.pictures }}</td>
-          <td>{{ lane.clubs }}</td>
-        </tr>
-    {% endfor %}
-    </table>
+{% if object.current_star %}
+<div style="display: flex; flex-wrap: wrap;">
+    <div id="3d-graph"></div>
 
-  <div id="3d-graph" style="margin: 1em;"></div>
+    <div style="margin: 1em;">
+        <p><a onclick="focus_node(get_node_from_id({{ object.id }}))">Reset on {{ object.get_display_name() }}</a></p>
+        <p>Self score: {{ object.current_star.mass }}</p>
+        <table style="width: initial;">
+            <tr>
+              <th></th>
+              <th>Citizen</th>
+              <th>Score</th>
+              <th>Distance</th>
+              <th>Family</th>
+              <th>Pictures</th>
+              <th>Clubs</th>
+            </tr>
+        {% for lane in lanes %}
+            <tr>
+              <td><a onclick="focus_node(get_node_from_id({{ lane.other_star_id }}))">Locate</a></td>
+              <td><a href="{{ url("galaxy:user", user_id=lane.other_star_id) }}">{{ lane.other_star_name }}</a></td>
+              <td>{{ lane.other_star_mass }}</td>
+              <td>{{ lane.distance }}</td>
+              <td>{{ lane.family }}</td>
+              <td>{{ lane.pictures }}</td>
+              <td>{{ lane.clubs }}</td>
+            </tr>
+        {% endfor %}
+        </table>
+    </div>
+</div>
+<p>#{{ object.current_star.galaxy }}#</p>
 {% else %}
   <p>This citizen has not yet joined the galaxy</p>
 {% endif %}
@@ -53,9 +58,31 @@
       return Graph.graphData().nodes.find(n => n.id === id);
   }
 
+  function get_links_from_node_id(id) {
+      return Graph.graphData().links.filter(l => l.source.id === id || l.target.id === id);
+  }
+
   function focus_node(node) {
+      highlightNodes.clear();
+      highlightLinks.clear();
+
+      hoverNode = node || null;
+      if (node) { // collect neighbors and links for highlighting
+        get_links_from_node_id(node.id).forEach(link => {
+          highlightLinks.add(link);
+          highlightNodes.add(link.source);
+          highlightNodes.add(link.target);
+        });
+      }
+
+      // refresh node and link display
+      Graph
+        .nodeThreeObject(Graph.nodeThreeObject())
+        .linkWidth(Graph.linkWidth())
+        .linkDirectionalParticles(Graph.linkDirectionalParticles());
+
       // Aim at node from outside it
-      const distance = 200;
+      const distance = 42;
       const distRatio = 1 + distance/Math.hypot(node.x, node.y, node.z);
 
       const newPos = node.x || node.y || node.z
@@ -69,25 +96,44 @@
       );
   }
 
+  const highlightNodes = new Set();
+  const highlightLinks = new Set();
+  let hoverNode = null;
+
   document.addEventListener("DOMContentLoaded", () => {
+    var graph_div = document.getElementById('3d-graph');
     Graph = ForceGraph3D();
-    Graph(document.getElementById('3d-graph'));
+    Graph(graph_div);
     Graph
         .jsonUrl('{{ url("galaxy:data") }}')
-        .width(1000)
-        .height(700)
-        .nodeAutoColorBy('id')
-        .nodeLabel(node => `${node.name}`)
-        .onNodeClick(node => focus_node(node))
-        .linkDirectionalParticles(3)
-        .linkDirectionalParticleWidth(0.8)
-        .linkDirectionalParticleSpeed(0.006)
+        .width(graph_div.parentElement.clientWidth > 1200 ? 1200 : graph_div.parentElement.clientWidth) // Not perfect at all. JS-fu master from the future, please fix this :-)
+        .height(1000)
+        .enableNodeDrag(false) // allow easier navigation
+        .onNodeClick(node => {
+             camera = Graph.cameraPosition();
+             var distance = Math.sqrt(Math.pow(node.x - camera.x, 2) + Math.pow(node.y - camera.y, 2) + Math.pow(node.z - camera.z, 2))
+             if (distance < 120 || highlightNodes.has(node)) {
+                 focus_node(node);
+             }
+         })
+        .linkWidth(link => highlightLinks.has(link) ? 0.4 : 0.0)
+        .linkColor(link => highlightLinks.has(link) ? 'rgba(255,160,0,1)' : 'rgba(128,255,255,0.6)')
+        .linkVisibility(link => highlightLinks.has(link))
+        .nodeVisibility(node => highlightNodes.has(node) || node.mass > 4)
+        // .linkDirectionalParticles(link => highlightLinks.has(link) ? 3 : 1) // kinda buggy for now, and slows this a bit, but would be great to help visualize lanes
+        .linkDirectionalParticleWidth(0.2)
+        .linkDirectionalParticleSpeed(-0.006)
         .nodeThreeObject(node => {
           const sprite = new SpriteText(node.name);
           sprite.material.depthWrite = false; // make sprite background transparent
-          sprite.color = node.color;
-          sprite.textHeight = 5;
+          sprite.color = highlightNodes.has(node) ? node === hoverNode ? 'rgba(200,0,0,1)' : 'rgba(255,160,0,0.8)' : 'rgba(0,255,255,0.2)';
+          sprite.textHeight = 2;
+          sprite.center = new THREE.Vector2(1.2, 0.5);
           return sprite;
+        })
+        .onEngineStop( () =>  {
+          focus_node(get_node_from_id({{ object.id }}));
+          Graph.onEngineStop(() => {}); // don't call ourselves in a loop while moving the focus
         });
 
     // Set distance between stars
@@ -98,9 +144,6 @@
     Graph.d3Force('positionX', d3.forceX().strength(node => { return 1 - (1 / node.mass); }));
     Graph.d3Force('positionY', d3.forceY().strength(node => { return 1 - (1 / node.mass); }));
     Graph.d3Force('positionZ', d3.forceZ().strength(node => { return 1 - (1 / node.mass); }));
-
-    // Focus current user
-    setTimeout(() => focus_node(get_node_from_id({{ object.id }})), 1000);
   })
   </script>
 {% endblock %}

galaxy/tests.py

@@ -22,14 +22,19 @@
 #
 #
 
-from django.test import TestCase
+import json
+
+from pathlib import Path
+
 from django.core.management import call_command
+from django.test import TestCase
+from django.urls import reverse
 
 from core.models import User
 from galaxy.models import Galaxy
 
 
-class GalaxyTest(TestCase):
+class GalaxyTestModel(TestCase):
     def setUp(self):
         self.root = User.objects.get(username="root")
         self.skia = User.objects.get(username="skia")
@@ -41,6 +46,9 @@ class GalaxyTest(TestCase):
         self.com = User.objects.get(username="comunity")
 
     def test_user_self_score(self):
+        """
+        Test that individual user scores are correct
+        """
         with self.assertNumQueries(8):
             self.assertEqual(Galaxy.compute_user_score(self.root), 9)
             self.assertEqual(Galaxy.compute_user_score(self.skia), 10)
@@ -52,6 +60,10 @@ class GalaxyTest(TestCase):
             self.assertEqual(Galaxy.compute_user_score(self.com), 1)
 
     def test_users_score(self):
+        """
+        Test on the default dataset generated by the `populate` command
+        that the relation scores are correct
+        """
         expected_scores = {
             "krophil": {
                 "comunity": {"clubs": 0, "family": 0, "pictures": 0, "score": 0},
@@ -112,33 +124,78 @@ class GalaxyTest(TestCase):
             while len(users) > 0:
                 user1 = users.pop(0)
                 for user2 in users:
-                    score, family, pictures, clubs = Galaxy.compute_users_score(
-                        user1, user2
-                    )
+                    score = Galaxy.compute_users_score(user1, user2)
                     u1 = computed_scores.get(user1.username, {})
                     u1[user2.username] = {
-                        "score": score,
-                        "family": family,
-                        "pictures": pictures,
-                        "clubs": clubs,
+                        "score": sum(score),
+                        "family": score.family,
+                        "pictures": score.pictures,
+                        "clubs": score.clubs,
                     }
                     computed_scores[user1.username] = u1
 
         self.maxDiff = None  # Yes, we want to see the diff if any
         self.assertDictEqual(expected_scores, computed_scores)
 
+    def test_rule(self):
+        """
+        Test on the default dataset generated by the `populate` command
+        that the number of queries to rule the galaxy is stable.
+        """
+        galaxy = Galaxy.objects.create()
+        with self.assertNumQueries(58):
+            galaxy.rule(0)  # We want everybody here
+
+
+class GalaxyTestView(TestCase):
+    @classmethod
+    def setUpTestData(cls):
+        """
+        Generate a plausible Galaxy once for every test
+        """
+        call_command("generate_galaxy_test_data", "-v", "0")
+        galaxy = Galaxy.objects.create()
+        galaxy.rule(26)  # We want a fast test
+
     def test_page_is_citizen(self):
-        Galaxy.rule()
+        """
+        Test that users can access the galaxy page of users who are citizens
+        """
         self.client.login(username="root", password="plop")
-        response = self.client.get("/galaxy/1/")
+        user = User.objects.get(last_name="n°500")
+        response = self.client.get(reverse("galaxy:user", args=[user.id]))
         self.assertContains(
             response,
-            '<a onclick="focus_node(get_node_from_id(8))">Locate</a>',
+            f'<a onclick="focus_node(get_node_from_id({user.id}))">Reset on {user}</a>',
             status_code=200,
         )
 
     def test_page_not_citizen(self):
-        Galaxy.rule()
+        """
+        Test that trying to access the galaxy page of a user who is not
+        citizens return a 404
+        """
         self.client.login(username="root", password="plop")
-        response = self.client.get("/galaxy/2/")
+        user = User.objects.get(last_name="n°1")
+        response = self.client.get(reverse("galaxy:user", args=[user.id]))
         self.assertEquals(response.status_code, 404)
+
+    def test_full_galaxy_state(self):
+        """
+        Test on the more complex dataset generated by the `generate_galaxy_test_data`
+        command that the relation scores are correct, and that the view exposes the
+        right data.
+        """
+        self.client.login(username="root", password="plop")
+        response = self.client.get(reverse("galaxy:data"))
+        state = response.json()
+
+        galaxy_dir = Path(__file__).parent
+
+        # Dump computed state, either for easier debugging, or to copy as new reference if changes are legit
+        (galaxy_dir / "test_galaxy_state.json").write_text(json.dumps(state))
+
+        self.assertEqual(
+            state,
+            json.loads((galaxy_dir / "ref_galaxy_state.json").read_text()),
+        )

galaxy/views.py

@@ -45,32 +45,29 @@ class GalaxyUserView(CanViewMixin, UserTabsMixin, DetailView):
 
     def get_object(self, *args, **kwargs):
         user: User = super(GalaxyUserView, self).get_object(*args, **kwargs)
-        if not hasattr(user, "galaxy_user"):
+        if user.current_star is None:
             raise Http404(_("This citizen has not yet joined the galaxy"))
         return user
 
-    def get_queryset(self):
-        return super(GalaxyUserView, self).get_queryset().select_related("galaxy_user")
-
     def get_context_data(self, **kwargs):
         kwargs = super(GalaxyUserView, self).get_context_data(**kwargs)
         kwargs["lanes"] = (
             GalaxyLane.objects.filter(
-                Q(star1=self.object.galaxy_user) | Q(star2=self.object.galaxy_user)
+                Q(star1=self.object.current_star) | Q(star2=self.object.current_star)
             )
             .order_by("distance")
             .annotate(
                 other_star_id=Case(
-                    When(star1=self.object.galaxy_user, then=F("star2__owner__id")),
+                    When(star1=self.object.current_star, then=F("star2__owner__id")),
                     default=F("star1__owner__id"),
                 ),
                 other_star_mass=Case(
-                    When(star1=self.object.galaxy_user, then=F("star2__mass")),
+                    When(star1=self.object.current_star, then=F("star2__mass")),
                     default=F("star1__mass"),
                 ),
                 other_star_name=Case(
                     When(
-                        star1=self.object.galaxy_user,
+                        star1=self.object.current_star,
                         then=Case(
                             When(
                                 star2__owner__nick_name=None,
@@ -101,4 +98,4 @@ class GalaxyUserView(CanViewMixin, UserTabsMixin, DetailView):
 
 class GalaxyDataView(FormerSubscriberMixin, View):
     def get(self, request, *args, **kwargs):
-        return JsonResponse(Galaxy.objects.first().state)
+        return JsonResponse(Galaxy.get_current_galaxy().state)