Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 664588 - Dynamic indexing causes elasticsearch to hit field limit.
Summary: Dynamic indexing causes elasticsearch to hit field limit.
Status: RESOLVED OBSOLETE
Alias: None
Product: Websites
Classification: Unclassified
Component: Packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Packages Website
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-08-25 15:45 UTC by Conrad Kostecki
Modified: 2020-05-26 07:48 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Conrad Kostecki gentoo-dev 2018-08-25 15:45:15 UTC
When looking at PGO, I can see the newest commits, but the new package version is not shown by PGO.

E.G.: https://packages.gentoo.org/packages/media-sound/teamspeak-server

Reproducible: Always
Comment 1 Alec Warner (RETIRED) archtester gentoo-dev Security 2018-08-25 16:46:13 UTC
Yeah I think Dynamic field detection in ES is causing the PUTs to fail.

I'll see if I can turn them off.

-A
Comment 2 Conrad Kostecki gentoo-dev 2018-09-02 14:29:07 UTC
Seems to be fixed, as current packages are now shown again on PGO.
Comment 3 Alec Warner (RETIRED) archtester gentoo-dev Security 2018-09-02 16:29:26 UTC
(In reply to Conrad Kostecki from comment #2)
> Seems to be fixed, as current packages are now shown again on PGO.

Not quite, still waiting to verify it doesn't come back.

-A
Comment 4 Conrad Kostecki gentoo-dev 2019-10-24 20:28:45 UTC
Seems to be still a problem, since I see from time to time an deprecated package list on p.g.o.
Comment 5 Conrad Kostecki gentoo-dev 2019-12-05 17:51:01 UTC
There seems to be currently some problems, as packages seems to be outdated.

For example: https://packages.gentoo.org/packages/games-simulation/openrct2
Comment 6 Michael 'veremitz' Everitt 2019-12-06 09:40:11 UTC
(In reply to Conrad Kostecki from comment #5)
> There seems to be currently some problems, as packages seems to be outdated.
> 
> For example: https://packages.gentoo.org/packages/games-simulation/openrct2

This isn't an unusual problem, but it seems the root cause(s) have never been fully traced. It needs someone with good knowledge of Ruby and ElasticSearch (or more free time, if this doesn't apply to Alec as interim maintainer).
Comment 7 Max Magorsch (RETIRED) Gentoo Infrastructure gentoo-dev 2019-12-22 00:28:00 UTC
Dynamic mapping has been completely turned off in #3c9eaa1. To make a long story short, the structure of the index had to be changed to avoid dynamic mapping. That's what happened in #ebd404d and #9fb7236 so that dynamic mapping is not needed anymore.

So the field limit should definitely not be hit anymore. Furthermore issues due to dynamic mapping should not be present anymore as dynamic mapping is turned of and an exception is thrown if new fields are detected. 

However, I am not completely sure whether this is already deployed to p.g.o and whether a fresh index has been populated already.
Comment 8 Alec Warner (RETIRED) archtester gentoo-dev Security 2020-05-26 07:48:37 UTC
We don't use ES anymore.

-A