You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently metadata of L0 log store is stored as TableChangeLog in hummock version, which will be propagated via the notification service.
Recently we have seen that then the metadata size of TableChangeLog is larger (e.g. due to 30-day subscription retention), the hummock version size can become several hundred MBs, which can cause inefficiency in hummock event handler (see the clone here) and potentially in other places.
Some ideas to improve it:
Don't store TableChangeLog in hummock version and exclude it from the notification. iter_log retrieves the TableChangeLog from meta on demand.
Keep TableChangeLog in hummock version and notification. But we can use a separate structure to store TableChangeLog in CN to avoid cloning the full TableChangLog on every new version delta.
The text was updated successfully, but these errors were encountered:
Currently metadata of L0 log store is stored as TableChangeLog in hummock version, which will be propagated via the notification service.
Recently we have seen that then the metadata size of
TableChangeLog
is larger (e.g. due to 30-day subscription retention), the hummock version size can become several hundred MBs, which can cause inefficiency in hummock event handler (see the clone here) and potentially in other places.Some ideas to improve it:
TableChangeLog
in hummock version and exclude it from the notification.iter_log
retrieves theTableChangeLog
from meta on demand.TableChangeLog
in hummock version and notification. But we can use a separate structure to storeTableChangeLog
in CN to avoid cloning the fullTableChangLog
on every new version delta.The text was updated successfully, but these errors were encountered: