Discussion:
[mongodb-user] MongoDB IN Virtual memory
Ketul
2015-03-30 04:53:06 UTC
Permalink
HI Sir,

I have 10GB Data in my mongodB Database and i have 4G RAM when i am going
to try to fetch some record then how much data will it takes memory and
give me result and i am not sure about how much memory required for large
database so i need information about that?
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/af106a59-d428-429d-998e-8e034d1f501e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Tim Hawkins
2015-03-30 05:33:04 UTC
Permalink
It depends on the size of your "working set". For example i have a news
site that stores articles, it has 20,000 articles online, but only 2500
are from the last 2 weeks. I consider the 2500 as being in my "working set"
as they are the set of articles that 90% of my users will access, and they
should be cached in memory. I also have a set of indexes on these articles,
so it should keep as much of those indexes in memory as possible. My
database is 13GB but includes all the text of the articles and the photos
in them. My indexes on my 20,000 articles is 800mb, So i estimate my
workingset size to be about 2500/20000 * 13gb which is 1.6GB+800mb . So my
system should fit comfortably inside a 4GB system.

I have seen a number of posts from you on this, you seem to be searching
for a magic formula that will give you an answer, the above is about as
good as you are going to get, but it requires you to know and understand
the nature of your data, how it will typicaly accessed, and only you can
answer that, my formula above works only for my system and my data, yours
will be different.
Post by Ketul
HI Sir,
I have 10GB Data in my mongodB Database and i have 4G RAM when i am going
to try to fetch some record then how much data will it takes memory and
give me result and i am not sure about how much memory required for large
database so i need information about that?
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/af106a59-d428-429d-998e-8e034d1f501e%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/af106a59-d428-429d-998e-8e034d1f501e%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAEbsasSZmPBPd_DvTPk5ZcmtBixxJatCS46YzFYJgpxbYLbhhw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-03-30 05:37:37 UTC
Permalink
Thanking to Reply me.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/adaa8406-287d-4942-999f-beb4330f3d4e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
s.molinari
2015-03-30 06:42:41 UTC
Permalink
Yes, thanks for the info Tim. This post is a bit off-topic, but I'd just
like to throw it in.

We are going with a base ratio of RAM to Storage of 1:10, for calculation
of our platform's business plan. That is to say, for every 10GB of stored
data, we'll need 1GB of RAM. We know the ratio can be wildly variable, but
we feel this is tight enough to be a valid basis. The reason why we also
need this "rule of thumb ratio" is, we need a "least common denominator" to
figure out pricing for the database service. The disk storage is the least
common denominator. Currently, we think we'll be at about $0.20 per MB of
data ($20 per GB). The only difference we have to other services is, the
database service is going to be part of the whole system. The measurement
of system performance will be overall latency for a response. Sort of
getting into too much detail.... at any rate....

You say you have your images also in the database. If I may ask, why did
you decide to go that route? We are planning to use cloud block storage and
a CDN for public images (and naturally, only if the customer wants to use
it too. If the customer chooses not to, the file data will be in a GridFS
instance and will cost considerably more to store....). It is a simple
logical choice though, having public image files on in block storage
connected to a CDN will save a lot on database resources, thus saving money
and at the same time speed up page delivery. So, that is why I was
wondering about your choice of storage method.

Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/45b4a135-3422-4f18-954e-ded0f01d8229%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-03-30 09:13:19 UTC
Permalink
Hi Tim,
Yes i got your point , but in my case , i have 22930268 documents , each
documents contains {]_id,API_key,message, channel} fields and now i want to
fetch record from that it will takes about 10 minutes
.
my query likes: db.history.find({apikey:dassfsfsf,channels:ssfgdfgdghhhgh})
here i ahve create index on api_key and channels still it takes lots of
time. so for that what i have to do so it will give me quick result.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/d2ca3b01-533b-452f-a3bb-7a55b73ea887%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Tim Hawkins
2015-03-30 09:33:39 UTC
Permalink
Check the cardinality of your keys, if you have 229930268 objects and
API_Key is the same on virtualy all objects, then the index contributes
nothing to the selection process. Your index keys have to be unique enough
to reduce the records traversed down to a smaller number. If your query
still selects a large percentage of the records then you will have a
performance issue.

Use the .explain() function on the end of your query to tell you which
index was used and how many documents where scanned ie

db.history.find({apikey:dassfsfsf,channels:ssfgdfgdghhhgh}).explain(). If
you have a very large number of scanned documents then you either have a
problem with your indexes, or you have a problem with the structure of your
data.
Post by Ketul
Hi Tim,
Yes i got your point , but in my case , i have 22930268 documents , each
documents contains {]_id,API_key,message, channel} fields and now i want to
fetch record from that it will takes about 10 minutes
.
db.history.find({apikey:dassfsfsf,channels:ssfgdfgdghhhgh})
here i ahve create index on api_key and channels still it takes lots of
time. so for that what i have to do so it will give me quick result.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/d2ca3b01-533b-452f-a3bb-7a55b73ea887%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/d2ca3b01-533b-452f-a3bb-7a55b73ea887%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAEbsasR5xT6roW3cbXSQJkC9367p5kwZjmd_oSg8h0wOzxeh8w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-03-30 09:42:45 UTC
Permalink
Hi Tim ,
Thanks , yes my api_key and Channels which i used as index which is not
unique
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/d2682167-45ed-449c-a1b6-ca07aba28d2a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-03-30 09:58:52 UTC
Permalink
Post by Ketul
Hi Tim ,
Thanks , yes my api_key and Channels which i used as index which is not
unique
db.history.find( {"App_Key" : "Mayur1","Channels":["a1","a2"]}).explain()

"cursor" : "BasicCursor",
"isMultiKey" : false,
"n" : 15,
"nscannedObjects" : 22911539,
"nscanned" : 22911539,
"nscannedObjectsAllPlans" : 22911539,
"nscannedAllPlans" : 22911539,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 922619,
"nChunkSkips" : 0,
"millis" : 373404,
"server" : "CI5W8P111402:27017",
"filterSet" : false
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/2efa049f-9379-4876-8291-5aa36925bb97%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Tim Hawkins
2015-03-30 11:31:30 UTC
Permalink
Its not using any indexes at all.

Can you do the following

db.history.find().limit(1).pretty()

db.history.getIndexes()
Post by Ketul
Post by Ketul
Hi Tim ,
Thanks , yes my api_key and Channels which i used as index which is
not unique
db.history.find( {"App_Key" : "Mayur1","Channels":["a1","a2"]}).explain()
"cursor" : "BasicCursor",
"isMultiKey" : false,
"n" : 15,
"nscannedObjects" : 22911539,
"nscanned" : 22911539,
"nscannedObjectsAllPlans" : 22911539,
"nscannedAllPlans" : 22911539,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 922619,
"nChunkSkips" : 0,
"millis" : 373404,
"server" : "CI5W8P111402:27017",
"filterSet" : false
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/2efa049f-9379-4876-8291-5aa36925bb97%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/2efa049f-9379-4876-8291-5aa36925bb97%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAEbsasT5FF4n6c4ZsG0mdq1pFyoOz7zp9ZaVa%2BV1FbmjaDUOuA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-03-30 12:34:27 UTC
Permalink
hi Tim
it will give me result like
db.history.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "yalgaar_clients_historical_data.history"
},
{
"v" : 1,
"key" : {
"App_Key" : 1
},
"name" : "App_Key_1",
"ns" : "yalgaar_clients_historical_data.history"
}
]
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/b45503e8-ce10-47d8-9b41-08ae3d2a5ae6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Tim Hawkins
2015-03-30 13:24:26 UTC
Permalink
need that schema dump with

db.history.find().limit(1).pretty()
hi Tim
it will give me result like
Post by Tim Hawkins
db.history.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "yalgaar_clients_historical_data.history"
},
{
"v" : 1,
"key" : {
"App_Key" : 1
},
"name" : "App_Key_1",
"ns" : "yalgaar_clients_historical_data.history"
}
]
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/b45503e8-ce10-47d8-9b41-08ae3d2a5ae6%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/b45503e8-ce10-47d8-9b41-08ae3d2a5ae6%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAEbsasR4hS%3DfUisGEEUcs7ZYvgyzhhw%2Bs-z%2B5pkMbh2Vh80P%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Tim Hawkins
2015-03-30 13:28:55 UTC
Permalink
can you also do

db.history.find( {"App_Key" : "Mayur1"}).count();
Post by Tim Hawkins
need that schema dump with
db.history.find().limit(1).pretty()
hi Tim
it will give me result like
Post by Tim Hawkins
db.history.getIndexes()
[
{
"v" : 1,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "yalgaar_clients_historical_data.history"
},
{
"v" : 1,
"key" : {
"App_Key" : 1
},
"name" : "App_Key_1",
"ns" : "yalgaar_clients_historical_data.history"
}
]
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/b45503e8-ce10-47d8-9b41-08ae3d2a5ae6%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/b45503e8-ce10-47d8-9b41-08ae3d2a5ae6%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAEbsasT-gA8zCZHxPCt8_c69N5A7zAp5NkwAzS5DtsrM-mJeYg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-03-31 06:57:01 UTC
Permalink
when I try to create index in App_Key and Channels then it's give me
following errors. so please me suggest what acttually happend here.
db.history.createIndex({"App_Key":1,"Channels":1})
{
"createdCollectionAutomatically" : false,
"numIndexesBefore" : 1,
"ok" : 0,
"errmsg" : "Btree::insert: key too large to index, failing
yalgaar_clients_historical_
039 { : \"ck-68bd0e859e398a16\", :
\"sssssssssssssssssssssssssssssssssssssssssssssssssssssssss
sssssssssssssssssssssssssssssssssssssssssssssssssssssssss...\" }",
"code" : 17282
}

thanx in advance
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/4f734ce1-93b8-4c1b-babd-cfe2928339b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
s.molinari
2015-03-31 07:26:04 UTC
Permalink
The error indicates that the key you are indexing on is more than 1KB in
size, which is a hard limit in MongoDB's indexing system.

http://docs.mongodb.org/manual/reference/limits/#Index-Key-Limit

Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/eea905f7-5354-4415-8251-17900cce31b9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-03-31 08:07:56 UTC
Permalink
Thanks Scott.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/f232c5ab-a613-45d0-93e4-ea0d471c4e08%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Tim Hawkins
2015-03-31 09:37:23 UTC
Permalink
I think its pretty certain, that because of the structure of the data and
the lack of a complete set of indexs, the server is tablescanning most if
not all of the documents in the query. Hence it is having to read most of
the data each time. Even when you have indexs, they have to be setup up so
the number of documents hit by the query is signiicantly reduced, in this
case as per the explain output you posted, that is not the case.
Post by Ketul
Thanks Scott.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/f232c5ab-a613-45d0-93e4-ea0d471c4e08%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/f232c5ab-a613-45d0-93e4-ea0d471c4e08%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAEbsasSP7oxgMfY5856VLprq6gRZ_716jY%3DtEPCDPMLqu3%3D_Vg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-03-31 11:37:07 UTC
Permalink
THank you so much Tim & Scott.

Now its work , i have create index on both field so its created and its
work, it gives quick result.
Now My question is
my database size increasing day by day due to that size of index also
increasing so one day its cross limit of memory size right?
so now what will happen, it read index from disk as limit size of RAM. yes?
then what i have to do ?

Increase RAM Size or go no some another solution?
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/6943e65f-1782-44f1-a162-23a1a6dee1f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
s.molinari
2015-03-31 13:08:35 UTC
Permalink
There are only two choices. Scale up or scale out. Since you said you only
have 4GB of RAM, you have plenty of room to scale up and I'd suggest that
be the first course of action. Later on, when you are filling a good
portion of a large server or your I/O starts to get critical, then you
should scale out. Sharding should be a last resort, simply because it adds
complexity.

Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/a045887f-c048-4d9e-97e6-ebf8870e3852%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-04-01 05:17:03 UTC
Permalink
Thanks Scott.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/27cd761f-e1ca-4486-86f6-7756a98aa2d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-04-01 05:26:14 UTC
Permalink
Hi Scott,

If i scale - out means adding more nodes so its same as sharding? as here
i am doing horizontlly scaliing by adding more node. Right?
U said in prevoius post that when database size growup or i/o starts to
get critical then i have to move on sharding or something.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/0746bc32-9f7b-4c67-bd11-8b2011dcc4b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
s.molinari
2015-04-01 05:34:18 UTC
Permalink
Yes, scaling out means using sharding.

Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/569caed9-cd17-4b0a-9156-7b38d839f52e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-04-01 05:38:34 UTC
Permalink
Thanks Scott.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/0fb57236-2db5-4e97-97a6-82d5b3c37313%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Tim Hawkins
2015-04-01 07:26:31 UTC
Permalink
For increasing query capacity, sharding is not the only option, you can use
a replica set and then have the language driver load balance queries across
the members of the replica set. You also get failover resiliance. You
should move to replica sets first, and then probaly only shard if you are
having storage capacity problems.
Post by s.molinari
There are only two choices. Scale up or scale out. Since you said you only
have 4GB of RAM, you have plenty of room to scale up and I'd suggest that
be the first course of action. Later on, when you are filling a good
portion of a large server or your I/O starts to get critical, then you
should scale out. Sharding should be a last resort, simply because it adds
complexity.
Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/a045887f-c048-4d9e-97e6-ebf8870e3852%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/a045887f-c048-4d9e-97e6-ebf8870e3852%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAEbsasToWvV3AbJC8ebS-17f9LwLz8cCCS82rNQ%3DZHy%2BgvN4pg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
s.molinari
2015-04-01 08:53:28 UTC
Permalink
Yes, I was assuming Ketul's production environment was already a replica
set, as is suggested by MongoDB.

As for scaling reads across replicas. Check out this great article from
Asya. <http://askasya.com/post/canreplicashelpscaling>

Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/55f2526d-13b4-4f97-bbb2-b1a68cef3650%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-04-02 10:47:38 UTC
Permalink
Hi Scott and Tim,
i am dam confused about Tiger wired and MMAPv1 storage engine ? can
please you please clear my concept about that.?
thank in advance
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/aea7e8ee-7053-4d4f-ae20-1445de8f1a4a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Tim Hawkins
2015-04-02 13:20:38 UTC
Permalink
There is plenty of online material about that, cant you google it try

'Mongodb mmap vs wiredtiger"
Post by Ketul
Hi Scott and Tim,
i am dam confused about Tiger wired and MMAPv1 storage engine ? can
please you please clear my concept about that.?
thank in advance
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups
"mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit
https://groups.google.com/d/msgid/mongodb-user/aea7e8ee-7053-4d4f-ae20-1445de8f1a4a%40googlegroups.com
<https://groups.google.com/d/msgid/mongodb-user/aea7e8ee-7053-4d4f-ae20-1445de8f1a4a%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/CAEbsasSoO%2BEE_gcs9BkgKPT2G0JR7_dMP64J8u%2BKPJ5NtCuBbA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
s.molinari
2015-04-02 17:50:19 UTC
Permalink
You're going to have to be a bit more specific. Confused about what? Do you
know what they are?

Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/aad43c13-f044-4fd7-b732-4499ec992719%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-04-06 09:26:42 UTC
Permalink
hi scott.

Replication can implements on another node ?
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/8bc87763-f411-4610-9cc8-0dfafa7187fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
s.molinari
2015-04-06 09:56:21 UTC
Permalink
Replication means you have multiple nodes. One is the primary, all others
are secondaries.

You really should read through the very good MonogDB documentation (as a
minimum) before asking questions like this.

The manual: https://docs.mongodb.org/manual/
A <https://docs.mongodb.org/manual/>bout Replica Sets:
https://docs.mongodb.org/manual/core/replication-introduction/

Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/11dfd2d5-cc00-484c-bedd-b5fff4f5ad1e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-04-06 10:19:04 UTC
Permalink
hie scoot,

i read it but i am still confisu about that, so i asked question..

Node is consider as one system? right?
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/89be9275-a762-44dc-b5d8-562c2a91be80%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
s.molinari
2015-04-06 10:59:23 UTC
Permalink
A node is a single mongod. A system (to me) would be a whole MongoDB
instance, which would hold either a single replica set (which hold multiple
nodes) or a shard of replica sets.

Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/a9ca4310-b806-459f-94bd-cae9208884db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-04-06 12:08:23 UTC
Permalink
okay then suppose my system down then how can i access dataabse?
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/16760d2d-db63-47ec-91d2-48021faf0e24%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-04-06 12:14:20 UTC
Permalink
One system can contains many node okay then all node at a time access
main-memory? if its then performance can not down?
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/7f44096a-6fdc-4716-bf23-0dbceee3b0df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
s.molinari
2015-04-06 12:40:22 UTC
Permalink
If all we are talking about is your 4GB machine, then that should only
contain one mongod, which is the primary node.You can still work with a
MongoDB database within that machine, but only for development purposes.
For instance, I have a VM on my computer with only 3GB of RAM and MongoDB
runs on it, but it is only for development purposes.

Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/afedca69-1953-453c-9422-ba15b71eb76a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ketul
2015-04-06 12:50:04 UTC
Permalink
if one system contains one primary node and two secondary node okay...this
is case of replication..now my question is when my system goes down then
how can i achieve fault-toleance because of my system goes down all three
node also goes down then i cant access that right...so for remove this
problem can i do ? one system have one primary node and other two system
contains secondary node then now primary node system goes down then one of
the seconday sytem which contains replica so using that we can get data
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/75426521-d117-4b04-b25e-85a229b9fab9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
s.molinari
2015-04-06 13:40:53 UTC
Permalink
If you are looking for fault tolerance for a whole datacenter, you'd
probably need a geographically distributed replica set.

http://docs.mongodb.org/manual/core/replica-set-architecture-geographically-distributed/

And if the data center with the primary is not going to be online any time
soon and you know that data is lost forever for whatever reason, you could
work with the secondaries in the secondary data center to build a new
instance (system). I believe only the replication lag you had at the time
the primary data center went down, would be any data lost. This is all
theoretical from me, as I've never done this before. Anyone with more
experience please correct me, if I am wrong.

Scott
--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+***@googlegroups.com.
To post to this group, send email to mongodb-***@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/06c80da6-e4f6-4676-ab32-f5abe1359b31%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Loading...