services: add StateFetcher service for state snapshot downloading from NeoFS #3844

AliceInHunterland · 2025-03-20T17:43:44Z

fix tests import cycle
documentation

AliceInHunterland · 2025-03-21T08:14:32Z

pkg/services/blockfetcher/blockfetcher.go

+
+// GetMPTNodes search for the state object and returns MPT nodes.
+func (bfs *Service) GetMPTNodes() ([][]byte, error) {
+	headerHeight := bfs.heightFunc() - 1


return if is shutdown

AliceInHunterland · 2025-03-21T08:35:58Z

pkg/network/server.go

@@ -902,6 +906,13 @@ func (s *Server) requestBlocksOrHeaders(p Peer) error {
 		return fmt.Errorf("%w: %w", errBlocksRequestFailed, err)
 	}
 	if requestMPTNodes {
+		if s.config.NeoFSStateSyncExtensions {
+			nodes, err := s.syncHeaderFetcher.GetMPTNodes()


not syncHeaderFetcher, but syncblockFetcher

pkg/network/server.go

pkg/services/blockfetcher/blockfetcher.go

AnnaShaleva · 2025-03-21T08:45:16Z

pkg/services/blockfetcher/blockfetcher.go

+	startIndex := bfs.heightFunc()/bfs.cfg.IndexFileSize + 1
+	for {
+		prm := client.PrmObjectSearch{}
+		filters := object.NewSearchFilters()
+		filters.AddFilter(bfs.cfg.IndexFileAttribute, fmt.Sprintf("%d", startIndex), object.MatchStringEqual)
+		filters.AddFilter("IndexSize", fmt.Sprintf("%d", bfs.cfg.IndexFileSize), object.MatchStringEqual)
+		prm.SetFilters(filters)
+
+		ctx, cancel := context.WithTimeout(bfs.ctx, bfs.cfg.Timeout)
+		blockOidsObject, err := bfs.objectSearch(ctx, prm)


After PR finalisation it will be useful to add a comment about this code to #3744, because it will be affected by index files support removal.

pkg/services/blockfetcher/blockfetcher.go

pkg/services/statefetcher/statefetcher.go

AnnaShaleva · 2025-03-31T08:08:04Z

@AliceInHunterland, what's the state of the current PR? Can I review?

AliceInHunterland · 2025-03-31T11:46:35Z

Tests not fixed and not tested with blocks with StateRootEnabled. Turning back to p2p sync after trying neofs sync is not yet implemented.

pkg/network/server.go

pkg/services/blockfetcher/blockfetcher.go

pkg/services/statefetcher/statefetcher.go

AnnaShaleva · 2025-04-07T14:21:10Z

pkg/services/statefetcher/statefetcher.go

+	var err error
+	ctx, cancel := context.WithTimeout(bfs.ctx, bfs.cfg.Timeout)
+	defer cancel()
+	for i := headerHeight; true; i-- {


We have already discussed that, we need to search for the nearest state sync point, not every block for sure.

AnnaShaleva · 2025-04-07T14:22:02Z

pkg/services/statefetcher/statefetcher.go

+	return rc, err
+}
+
+func (bfs *Service) retry(action func() error) error {


It's also the common code. This code should be placed to the same place where common fetchers' settings are handled, and then reused.

AnnaShaleva · 2025-04-07T14:23:20Z

pkg/services/statefetcher/statefetcher.go

+	bfs.stateModule.Store.Seek(storage.SeekRange{Prefix: tempPrefix}, func(k, v []byte) bool {
+		newKey := append(permPrefix, k[1:]...)
+		cache.Put(newKey, v)
+		cache.Delete(k)
+		return true


See how it's handled by StateSync module: firstly remove all items stored by the old key, then replace the active key from the old one to the new one, that's it.

AnnaShaleva · 2025-04-07T14:24:34Z

pkg/services/statefetcher/statefetcher.go

+		return true
+	})
+
+	if n, err := cache.PersistSync(); err != nil {


Not a single persist per the whole batch for sure. It should be batched persist. Have you seen the same pattern somewhere else?

The general idea is that this code must be robust to node shutdown.

AnnaShaleva

Tests/linter are failing. We also need an extension of the default node's configuration.

pkg/network/server.go

AnnaShaleva

The main point is: since we decide to move storage items based state synchronisation to StateSync Module, it should be completely Module's duty to properly update storage/MPT, store intermediate synchronisation state and properly recover from this state after the node restart. StateFetcher should only manage storage items downloading from the NeoFS.

So all code is written, but it should be properly organized and move to proper places, to keep StateSync module abstracted from storage items source. See the relevant comments in statesync module and statefetcher before the refactoring.

AnnaShaleva · 2025-04-15T10:01:25Z

pkg/config/application_config.go

+	if err := a.NeoFSStateFetcher.NeoFSService.Validate(); err != nil {
+		a.NeoFSStateFetcher.NeoFSService = a.NeoFSBlockFetcher.NeoFSService
+	}
+	if err := a.NeoFSStateFetcher.Validate(); err != nil {


Explicitly check if NeoFSStateFetcher contains empty container parameters and fill in from NeoFSBlockFetcher section prior to NeoFSStateFetcher validation.

AnnaShaleva · 2025-04-15T10:04:20Z

pkg/config/neofsstorage_config.go

@@ -0,0 +1,95 @@
+package config


s/pkg/config/neofsstorage_config.go/pkg/config/neofsservice_config.go

AnnaShaleva · 2025-04-15T10:07:38Z

pkg/config/neofsstorage_config.go

+	cid "github.com/nspcc-dev/neofs-sdk-go/container/id"
+)
+
+// NeoFSService represents the configuration for services connected with NeoFS.


services connected with NeoFS.

services interacting with NeoFS block/state storage

AnnaShaleva · 2025-04-15T10:10:09Z

pkg/config/neofsstorage_config.go

+
+// Equals allows to compare two NeoFSService instances, returns true if
+// they're equal.
+func (cfg *NeoFSService) Equals(o NeoFSService) bool {


This method should be used to compare two application config instances. Right now it's unused.

AnnaShaleva · 2025-04-15T10:13:53Z

pkg/config/neofsstorage_config.go

+}
+
+// NeoFSStateFetcher represents the configuration for the NeoFS StateFetcher service.
+// if NeoFSService configuration is omitted the NeoFSBlockFetcher.NeoFSService will be used.


if NeoFSService configuration is omitted the NeoFSBlockFetcher.NeoFSService will be used.

This comment is not related to the structure documentation. I'd move it either to ApplicationConfiguration.Validate() or to ApplicationConfiguration.NeoFSStateFetcher.

pkg/network/server.go

AnnaShaleva · 2025-04-15T10:57:57Z

pkg/network/server.go

@@ -394,6 +406,7 @@ func (s *Server) Shutdown() {
 func (s *Server) stateSyncCallBack() {
 	needHeaders := s.stateSync.NeedHeaders()
 	needBlocks := s.stateSync.NeedBlocks()
+	needMPTs := s.stateSync.NeedMPTNodes()


And given the fact that starting from this commit, statesync Module is able to use both MPT nodes and raw storage items, I'd rename NeedMPTNodes to some NeedStorageData or NeedContractStorageData in a separate commit.

AnnaShaleva · 2025-04-15T10:59:14Z

pkg/network/server.go

-	for p := range s.peers {
-		if p.Handshaked() {
-			heights = append(heights, p.LastBlockIndex())
+	if !s.ServerConfig.NeoFSBlockFetcherCfg.Enabled || s.syncStateFetcher.IsShutdown() {


This part needs some polishing, but let's firstly deal with final statesync Module implementation and after that I'll review network part one more time.

AnnaShaleva · 2025-04-15T11:01:31Z

pkg/services/statefetcher/statefetcher.go

+			if err := sfs.saveCheckpointState(tempState); err != nil {
+				sfs.log.Error("failed to save state checkpoint", zap.Error(err))
+			}


This part should be managed by statesync module (see the related reviewcomments in statesync module file). StateFetcher should only manage storage items downloading and should then pass these raw storage items to statesync module (exactly like P2P layer passes raw MPT data to statesync Module).

AnnaShaleva · 2025-04-15T11:02:33Z

pkg/services/statefetcher/statefetcher.go

+				if err := sfs.chain.AddMPTBatch(batch, syncHeight); err != nil {
+					sfs.log.Error("failed to process final batch during shutdown", zap.Error(err))
+				}
+				if _, err := localMPT.PutBatch(mpt.MapToMPTBatch(batch)); err != nil {
+					sfs.log.Error("failed to update local MPT during shutdown", zap.Error(err))
+				}
+				localMPT.Flush(syncHeight)
+			}


And this part should also be managed by StateSync Module. I'm sure I saw the similar code in the module, so there shouldn't be duplicating functionality between StateFetcher and Module.

codecov · 2025-04-16T13:52:43Z

Codecov Report

Attention: Patch coverage is 17.48148% with 557 lines in your changes missing coverage. Please review.

Project coverage is 81.66%. Comparing base (2bdc778) to head (13f91b2).
Report is 28 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/services/statefetcher/statefetcher.go	3.30%	234 Missing ⚠️
pkg/core/statesync/module.go	20.00%	198 Missing and 6 partials ⚠️
pkg/services/helpers/neofs/blockstorage.go	0.00%	62 Missing ⚠️
pkg/network/server.go	50.00%	27 Missing and 3 partials ⚠️
cli/util/upload_state.go	0.00%	18 Missing ⚠️
pkg/config/application_config.go	0.00%	4 Missing and 2 partials ⚠️
pkg/config/neofsservice_config.go	89.65%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3844      +/-   ##
==========================================
- Coverage   82.52%   81.66%   -0.86%     
==========================================
  Files         342      343       +1     
  Lines       47887    48598     +711     
==========================================
+ Hits        39518    39687     +169     
- Misses       6734     7266     +532     
- Partials     1635     1645      +10

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Close #3793 Signed-off-by: Ekaterina Pavlova <[email protected]>

AliceInHunterland force-pushed the statefetcher branch 2 times, most recently from b000eb3 to e39c22a Compare March 21, 2025 06:57

AliceInHunterland commented Mar 21, 2025

View reviewed changes

AnnaShaleva reviewed Mar 21, 2025

View reviewed changes

AliceInHunterland force-pushed the statefetcher branch from e39c22a to ea9e2ee Compare March 27, 2025 12:21

AliceInHunterland changed the title ~~blockfetcher: add GetMPTNodes~~ services: add StateFetcher service for state snapshot downloading from NeoFS Mar 27, 2025

AliceInHunterland requested a review from AnnaShaleva March 27, 2025 12:22

AliceInHunterland commented Mar 27, 2025

View reviewed changes

pkg/services/statefetcher/statefetcher.go Outdated Show resolved Hide resolved

AnnaShaleva requested changes Mar 27, 2025

View reviewed changes

pkg/services/statefetcher/statefetcher.go Outdated Show resolved Hide resolved

pkg/services/statefetcher/statefetcher.go Outdated Show resolved Hide resolved

pkg/services/statefetcher/statefetcher.go Outdated Show resolved Hide resolved

AliceInHunterland force-pushed the statefetcher branch from ea9e2ee to 11a2e66 Compare March 28, 2025 09:54

AnnaShaleva reviewed Mar 28, 2025

View reviewed changes

pkg/services/statefetcher/statefetcher.go Outdated Show resolved Hide resolved

AliceInHunterland force-pushed the statefetcher branch 4 times, most recently from f93aa3c to 2c617ce Compare March 28, 2025 13:10

AliceInHunterland force-pushed the statefetcher branch from 2c617ce to 0fa015c Compare March 31, 2025 11:44

AliceInHunterland requested a review from AnnaShaleva March 31, 2025 11:46

AnnaShaleva requested changes Apr 7, 2025

View reviewed changes

AnnaShaleva reviewed Apr 7, 2025

View reviewed changes

AliceInHunterland force-pushed the statefetcher branch 3 times, most recently from 9c45630 to ef0500d Compare April 14, 2025 07:59

AliceInHunterland requested a review from AnnaShaleva April 14, 2025 08:15

AliceInHunterland commented Apr 14, 2025

View reviewed changes

pkg/network/server.go Outdated Show resolved Hide resolved

AliceInHunterland force-pushed the statefetcher branch 2 times, most recently from 719d675 to 9b1616f Compare April 15, 2025 07:36

AliceInHunterland marked this pull request as ready for review April 15, 2025 08:09

AliceInHunterland force-pushed the statefetcher branch from 9b1616f to c9ac8bd Compare April 15, 2025 08:10

AliceInHunterland requested a review from roman-khimov April 15, 2025 08:11

AnnaShaleva reviewed Apr 15, 2025

View reviewed changes

AliceInHunterland force-pushed the statefetcher branch 2 times, most recently from 3d603a2 to 7beed76 Compare April 16, 2025 13:42

AliceInHunterland requested a review from AnnaShaleva April 16, 2025 13:47

services: add statefetcher service for NeoFS-based state sync

13f91b2

Close #3793 Signed-off-by: Ekaterina Pavlova <[email protected]>

AliceInHunterland force-pushed the statefetcher branch from 7beed76 to 13f91b2 Compare April 16, 2025 14:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

services: add StateFetcher service for state snapshot downloading from NeoFS #3844

services: add StateFetcher service for state snapshot downloading from NeoFS #3844

AliceInHunterland commented Mar 20, 2025 •

edited

Loading

AliceInHunterland Mar 21, 2025 •

edited

Loading

AliceInHunterland Mar 21, 2025 •

edited

Loading

AnnaShaleva Mar 21, 2025

AnnaShaleva commented Mar 31, 2025 •

edited

Loading

AliceInHunterland commented Mar 31, 2025 •

edited

Loading

AnnaShaleva Apr 7, 2025

AnnaShaleva Apr 7, 2025

AnnaShaleva Apr 7, 2025

AnnaShaleva Apr 7, 2025

AnnaShaleva left a comment

AnnaShaleva left a comment

AnnaShaleva Apr 15, 2025

AnnaShaleva Apr 15, 2025

AnnaShaleva Apr 15, 2025

AnnaShaleva Apr 15, 2025

AnnaShaleva Apr 15, 2025

AnnaShaleva Apr 15, 2025

AnnaShaleva Apr 15, 2025

AnnaShaleva Apr 15, 2025

AnnaShaleva Apr 15, 2025

codecov bot commented Apr 16, 2025 •

edited

Loading

services: add StateFetcher service for state snapshot downloading from NeoFS #3844

Are you sure you want to change the base?

services: add StateFetcher service for state snapshot downloading from NeoFS #3844

Conversation

AliceInHunterland commented Mar 20, 2025 • edited Loading

AliceInHunterland Mar 21, 2025 • edited Loading

Choose a reason for hiding this comment

AliceInHunterland Mar 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AnnaShaleva commented Mar 31, 2025 • edited Loading

AliceInHunterland commented Mar 31, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AnnaShaleva left a comment

Choose a reason for hiding this comment

AnnaShaleva left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Apr 16, 2025 • edited Loading

Codecov Report

AliceInHunterland commented Mar 20, 2025 •

edited

Loading

AliceInHunterland Mar 21, 2025 •

edited

Loading

AliceInHunterland Mar 21, 2025 •

edited

Loading

AnnaShaleva commented Mar 31, 2025 •

edited

Loading

AliceInHunterland commented Mar 31, 2025 •

edited

Loading

codecov bot commented Apr 16, 2025 •

edited

Loading