Miroir du site de l'AE UTBM https://ae.utbm.fr
Go to file
Skia 782cd9a45a Merge branch 'sexy-search' into 'master'
Sexy search

The goal of this MR is to solve the search issue #96. Let's assume we have a user with firstname `Jean-François`, lastname `Du Pont` and nickname `Ai'gnan`. Here is a list of search that did not include him previously but now includes him (was and still is case-insensitive):

* `jean françois` (missing -) ;
* `jean-francois` (missing ç) ;
* `jean francois` (both) ;
* `dupont` (space) ;
* `françois` (not the start of his name) ;
* `aignan` (missing ').

You get it, there are a lot of mistakes that humans can do. It also sorts results by `User.last_update` to avoid putting old accounts at the top of common requests (such as firstname-only or lastname-only requests).

### How it works

For those who don't know, the search is handled by Xapian (the search backend) through the haystack library which provides a Django-friendly interface to multiple search backends. Xapian maintains kind of a duplicate of the database (only for models against which we want to search something) which is optimised for search operations. Its "models" are called "indexes" (see `core.search_indexes.UserIndex` for the user model).

Every time a user is created or modified, it is indexed (through a signal handler) so that Xapian knows about it. For the user search, what is indexed is the string outputted by the `core/templates/search/indexes/core/user_auto.txt` template. For our example from above, it looks like this:

```
jean francois
du pont
aignan
jeanfrancois
dupont

jeanfrancoisdupont
```

As you can see, unicode is removed. There also are kind-of duplicates with different spacing as we are using an autocomplete algorithm: it searches from the beginning of words.

The one I am not sure about is the last one. Its goal is to allow searching without putting a space between the firstname and lastname. Is this useful?

The prod will have to do a `./manage.py update_index`, not sure it does it in the upgrade script.

See merge request ae/Sith!269
2021-09-28 00:14:38 +00:00
accounting Black update 2021-09-26 13:58:39 +02:00
api Upgrade black and format accordingly 2020-08-27 15:59:42 +02:00
club Upgrade black and format accordingly 2020-08-27 15:59:42 +02:00
com Black update 2021-09-26 13:58:39 +02:00
core search: reduce user index size 2021-09-28 01:44:15 +02:00
counter Black update 2021-09-26 13:58:39 +02:00
doc core: add ./manage.py check_front command and call it on runserver 2021-09-27 22:00:36 +02:00
eboutic Black update 2021-09-26 13:58:39 +02:00
election Black update 2021-09-26 13:58:39 +02:00
forum forum: fix ForumTopicSubscribeView error 500 with anonymous user 2019-12-16 15:00:33 +01:00
launderette Black update 2021-09-26 13:58:39 +02:00
locale/fr/LC_MESSAGES Added new subscription for the new CA offer 2021-09-28 01:11:23 +02:00
matmat django2.2: adapt ChoiceField to the new API 2019-10-08 22:46:02 +02:00
pedagogy pedagogy: uv autofill finishing touches 2019-10-24 14:18:29 +02:00
rootplace Upgrade black and format accordingly 2020-08-27 15:59:42 +02:00
sas sas: fix 500 error when tagging the same user twice or adding a non existing user 2019-12-17 11:25:17 +01:00
sith Added new subscription for the new CA offer 2021-09-28 01:11:23 +02:00
stock Upgrade black and format accordingly 2020-08-27 15:59:42 +02:00
subscription Added new subscription for the new CA offer 2021-09-28 01:11:23 +02:00
trombi trombi: raw tool for trombi admins to add a club membership to a trombi member 2020-03-22 16:14:37 +01:00
.coveragerc coverage: blacklist some files 2017-09-07 11:50:05 +02:00
.envrc documentation: add instructions for direnv 2019-10-08 00:59:19 +02:00
.gitattributes Add mappings for .po and .jinja files 2019-04-22 18:20:22 +02:00
.gitignore documentation: base structure for new documentation 2019-10-07 23:51:35 +02:00
.gitlab-ci.yml documentation: add doc compile test in pipeline 2019-10-07 23:51:38 +02:00
.readthedocs.yml documentation: base structure for new documentation 2019-10-07 23:51:35 +02:00
CONTRIBUTING.rst documentation: add CONTRIBUTING 2019-10-08 11:08:47 +02:00
LICENSE Apply GPLv3 to Sith, and add header to every concerned file 2017-04-24 17:51:12 +02:00
LICENSE.old Apply GPLv3 to Sith, and add header to every concerned file 2017-04-24 17:51:12 +02:00
manage.py Apply GPLv3 to Sith, and add header to every concerned file 2017-04-24 17:51:12 +02:00
migrate.py pedagogy: fix typo for AUTUMN_AND_SPRING 2019-07-05 16:40:02 +02:00
README.rst documentation: add help ressources and update installation instructions 2019-11-20 17:03:18 +01:00
requirements.txt django2.2: unlock djangorestframework version limit 2019-10-08 22:46:38 +02:00

.. image:: https://ae-dev.utbm.fr/ae/Sith/badges/master/pipeline.svg
  :target: https://ae-dev.utbm.fr/ae/Sith/commits/master
  :alt: pipeline status

.. image:: https://readthedocs.org/projects/sith-ae/badge/?version=latest
  :target: https://sith-ae.readthedocs.io/?badge=latest
  :alt: documentation Status

.. image:: https://ae-dev.utbm.fr/ae/Sith/badges/master/coverage.svg
  :target: https://ae-dev.utbm.fr/ae/Sith/commits/master
  :alt: coverage report

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
  :target: https://github.com/ambv/black
  :alt: code style: black

.. image:: https://img.shields.io/badge/zulip-join_chat-brightgreen.svg
  :target: https://ae-dev.zulipchat.com
  :alt: project chat

This is the source code of the UTBM's student association available at https://ae.utbm.fr/.

All documentation is in the ``docs`` directory and online at https://sith-ae.readthedocs.io/. This documentation is written in French because it targets a French audience and it's too much work to maintain two versions. The code and code comments are strictly written in English.

If you want to contribute, here's how we recommend to read the docs:

* First, it's advised to read the about part of the project to understand the goals and the mindset of the current and previous maintainers and know what to expect to learn.
* If in the first part you realize that you need more background about what we use, we provide some links to tutorials and documentation at the end of our documentation. Feel free to use it and complete it with what you found helpful.
* Keep in mind that this documentation is thought to be read in order.

To join our team :

* Send a mail at mailto:ae.utbm.fr
* Join our group chat at https://ae-dev.zulipchat.com
* See and join our Trello at https://trello.com/b/YQOaF33m/site-ae.

This project is licenced under GNU GPL, see the LICENSE file at the top of the repository for more details.