About writing data more than 500 in firebase with flutter

Issue

i have problem when i try to make write with batch.set into firebase database .. when i use small data to set it! its created correctly and so fast but when i write alot of data like 200 questions and try to updated by those codes! show to me error as limit cannot be exceed more than 500 as you can see down below

W/Firestore( 5494): (24.0.2) [WriteStream]: (13b1114) Stream closed with status: Status{code=INVALID_ARGUMENT, description=maximum 500 writes allowed per request, cause=null}.

those are codes that i try to set data into firebase:

import 'dart:convert';

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:get/get.dart';
import 'package:quizzle/firebase/firebase_configs.dart';
import 'package:quizzle/models/quiz_paper_model.dart';
import 'package:quizzle/utils/logger.dart';

const String folderName = '/assets/DB/papers';

class PapersDataUploader extends GetxController {
  @override
  void onReady() {
    uploadData();
    super.onReady();
  }

  final loadingStatus = LoadingStatus.loading.obs;

  uploadData() async {
    loadingStatus.value = LoadingStatus.loading; 
    final fi = FirebaseFirestore.instance;

    try {
      //read asset folder
      final manifestContent = await DefaultAssetBundle.of(Get.context!)
          .loadString('AssetManifest.json');
      final Map<String, dynamic> manifestMap = json.decode(manifestContent);
      //seperate quiz json files
      final papersInAsset = manifestMap.keys
          .where((path) =>
              path.startsWith('assets/DB/papers/') && path.contains('.json'))
          .toList();

      final List<QuizPaperModel> quizPapers = [];

      for (var paper in papersInAsset) {
        //read content of papers(json files)
        String stringPaperContent = await rootBundle.loadString(paper);
        //add data to model
        quizPapers.add(QuizPaperModel.fromString(stringPaperContent));
      }

      //upload data to firebase

      var batch = fi.batch();

      for (var paper in quizPapers) {
        batch.set(quizePaperFR.doc(paper.id), {
          "title": paper.title,
          "image_url": paper.imageUrl,
          "Description": paper.description,
          "time_seconds": paper.timeSeconds,
          "questions_count" : paper.questions == null ? 0 : paper.questions!.length
        }, 
        
        );

        for (var questions in paper.questions!) {
          
          final questionPath = questionsFR(
            paperId: paper.id,
            questionsId: questions.id
          );

          batch.set(questionPath, {
            "question": questions.question,
            "imageQue": questions.imageQue,
            "correct_answer": questions.correctAnswer
          });

          for (var answer in questions.answers) {
            batch.set(questionPath.collection('answers').doc(answer.identifier), {"identifier": answer.identifier, "answer": answer.answer});
          }
        }
      }
      await batch.commit();
      loadingStatus.value = LoadingStatus.completed;
    } on Exception catch (e) {
      AppLogger.e(e);
    }
  }
}

and this is as references:

import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:firebase_storage/firebase_storage.dart';

final fi = FirebaseFirestore.instance;

//FR - firestore reference

final userFR = fi.collection('users');
final quizePaperFR = fi.collection('quizpapers');
final leaderBoardFR = fi.collection('leaderboard');

DocumentReference recentQuizesData({required String userId, required String paperId}) => userFR.doc(userId).collection('myrecent_quizes').doc(paperId);

CollectionReference<Map<String, dynamic>> recentQuizes({required String userId}) => userFR.doc(userId).collection('myrecent_quizes');

CollectionReference<Map<String, dynamic>> getleaderBoard({required String paperId}) => leaderBoardFR.doc(paperId).collection('scores');

DocumentReference questionsFR({required String paperId, required String questionsId}) => quizePaperFR.doc(paperId).collection('questions').doc(questionsId);


Reference get firebaseStorage => FirebaseStorage.instance.ref();

and this as sample of data with multiple of files that i try to create it in firebase database:

{
    "id": "ppr001",
    "title": "PSA",
    "image_url": "",
    "Description": "CHAPTER 1: Analgesia, Anesthesia, and Procedural Sedation Questions",
    "time_seconds": 5580,
    "questions": [
        {
            "id": "ppr001q001",
            "question": "Which of the following is the recommended treatment for migraines?",
            "imageQue": "",
            "answers": [
                {
                    "identifier": "A",
                    "Answer": "Fentanyl (Sublimaze)"
                },
                {
                    "identifier": "B",
                    "Answer": "Hydrocodone and acetaminophen (Vicodin)"
                },
                {
                    "identifier": "C",
                    "Answer": "Meperidine hydrochloride (Demerol)"
                },
                {
                    "identifier": "D",
                    "Answer": "Sumatriptan succinate (Imitrex)"
                },
                {
                    "identifier": "E",
                    "Answer": "Morphine sulfate"
                }
            ],
            "correct_answer": "D"
        }
    ]
}

still I’m learning flutter as beginner .. so i hope someone can help me with those codes above to updated my database so easily each time even have more than 500 writes to added so easily in firebase !

thanks in advance

Solution

What you’ll want to do is keep a count of the writes you added to the batch, and then commit the batch once you get close to the limit.

var batch = fi.batch();
var count = 0; // 👈

for (var paper in quizPapers) {
  count++; // 👈
  batch.set(quizePaperFR.doc(paper.id), {
    "title": paper.title,
    "image_url": paper.imageUrl,
    "Description": paper.description,
    "time_seconds": paper.timeSeconds,
    "questions_count" : paper.questions == null ? 0 : paper.questions!.length
  }, 
  
  );

  for (var questions in paper.questions!) {      
    final questionPath = questionsFR(
      paperId: paper.id,
      questionsId: questions.id
    );

    count++; // 👈
    batch.set(questionPath, {
      "question": questions.question,
      "imageQue": questions.imageQue,
      "correct_answer": questions.correctAnswer
    });

    for (var answer in questions.answers) {
      batch.set(questionPath.collection('answers').doc(answer.identifier), {"identifier": answer.identifier, "answer": answer.answer});
      // 👇 If we get close to the limit, commit the batch and start a new one
      if (count++ > 450) {
        await batch.commit();
        batch = fi.batch();
        count = 0;
      }
    }
  }

}
await batch.commit();

If there are many questions without answers, you may need to repeat the blocks that check the count for other writes too.

Answered By – Frank van Puffelen

Answer Checked By – Cary Denson (FlutterFixes Admin)

Leave a Reply

Your email address will not be published. Required fields are marked *