Greetings.
Some of you may have noticed performance issues with slow git clones or
pushes or trac pages loading slowly recently.
On 2013-03-18 we found that our glusterfs replication was not working
as expected and attempted to correct it. Unfortunately the cluster was
not able to both replicate everything to catch up and also serve
requests. We have disabled replication again, but load remains at
unacceptable levels.
Note that these performance issues should not result in any data loss.
We are planning the following steps to alleviate / solve the issues:
1. In the next week, there will be an outage while we reconfigure
storage to increase performance. Hopefully after this change load
issues will be alleviated. An outage notification will be sent to this
list when we have the outage scheduled.
2. We are busy planning a fedorahosted version2 setup that splits out
projects to separate instances and allows much better scaling ability.
Once we have a plan in place, we will ask projects if they wish to
migrate to the new setup and help us test any issues.
We apologize for issues anyone has run into in the last few days.
Rest assured we are working on it.
kevin
