MongoDB Performance for more data than memory

I’ve recently been having a play around with MongoDB and it’s really cool. One of the common messages I see all over is that you should only use it if your dataset fits into memory. I’ve not yet seen any benchmarks on what happens when it doesn’t though. So, here is a benchmark on how MongoDB performs when the data is bigger than the amount of memory available.

Setup

Mongo server: An EC2 large instance (64 bit) running a Ubuntu 10.10 image from Alestic. Has 7.5gb of memory. Data folder was on the instance and not EBS.

Mongo client: An EC2 small instance.

Test

The test will involve inserting X documents to MongoDB with the following structure:

key: n (where n is 0, 1, … X – 1)
text: ‘Mary had a little lamb. ‘ x 100
There will be an index on key to prevent full scans of the data.

After the insert there will be 30,000 gets with random keys.

The expectation is that when the data set gets too large to fit in memory the random gets will become very slow. This will be due to MongoDB’s memory mapped files no longer fitting in memory and needing to be read from disk.

When this thrashing of the disk starts happening it will be interesting to see what happens when a subset of the dataset is read from. To investigate this a further test will be run that:

99% of the time – reads from a random key chosen from only Y% of the keys
1% of the time – reads from any key chosen from the entire dataset
The expectation here is that for small Y the performance will be similar to when the entire dataset is in memory – as the pages that contain the subset of data will be in memory already and not need to read from disk.

Results

Basic results

A result spreadsheet is available here (Google Doc).

Up to 3 million documents the reads were consistent around 17s for 30,000 reads:

      Keys  Average time (s)  Memory usage (mb)
    10,000              16.8   forgot to check
   100,000              16.9               547
 1,000,000              18.0              1672
 3,000,000              17.2              4158
10,000,000              74.1              7469 (16.1gb inc. virtual)

Once the dataset got larger than the amount of memory available the read time got slow. It wasn’t as slow as it could be in extreme cases as roughly half of the dataset would still have been in memory.
It’s worth noting that at this point inserts started getting slow: 178s for 3 million documents vs 1,102s for 10 million documents (~17k inserts/sec vs ~9k inserts/sec).

What about when reading a subset more often?

Focus (%) Read 1 (s) Read 2 (s) Read 3 (s)
100 73.1 75.3 73.9
10 54.3 37.0 29.5
1 21.1 18.8 18.2
Focus in the above results refers to the %age of the dataset that was chosen for 99% of reads. In this case it was the first Y% of rows to be inserted – meaning that the pages were likely now out of memory by the time we wanted to read them.

The results show that MongoDB will perform just as fast on a dataset that is too large for memory if a small subset of the data is read from more frequently than the rest.

It was interesting to see the 10% figure drop over time. I suspect that this figure will get closer to 18s as the number of reads increases – more and more of the pages will be cached by the operating system and not need to be read from disk.

Conclusions

From doing this it can be seen that the performance of MongoDB can drop by an order of magnitude when the dataset gets too big for memory. However, if the reads are clustered in a subset of the dataset then a large amount of the data will be able to be kept in cache and reads kept quick.

It’s definitely worth noting that it’s normal for the performance to drop by an order of magnitude when the database has to start hitting disk. The point of this experiment was to make sure that it was only one order of magnitude and that if reads were focussed the performance would stay high.

Code

The code for the benchmark (for improvements and your own testing) is in github: http://github.com/colinhowe/mongo-benchmarks/blob/master/bench.py

Don’t confuse Ruby’s ‘throw’ statement with ‘raise’

Programmers who come to Ruby from other languages — particularly C++, C♯ or Java — tend to misuse Ruby’s throw statement. They type throw when they mean to type raise.

What’s the difference?

Like C++, C♯ and Java, Ruby has first-class support for exceptions. Exceptions can be thrown using the raise statement and caught using the rescue keyword. Rubyists often speak of “errors” rather than “exceptions.” This is because it’s accepted that the names of Ruby exception classes generally end in Error rather than Exception, excepting the base class for exceptions which is called Exception. (Still with me?)

You may say, “Okay, so I’ll start saying ‘error’ when I mean ‘exception.’ But why can’t I at least type the throw and catch keywords I’m used to typing?”

Ruby has a distinctive language feature that consists of pairs of throw and catch statements. The throw-catch paradigm works similarly to raise and rescue. When Ruby encounters a throw statement, like a good matchmaker, she walks back up the execution stack (“catch me a catch!”) looking for a suitable catch. Here’s some code to illustrate.

class Tevye

  def live
    sunrise
    sunset
    l_chaim!
    toil_in_anatevka
    :pauper
  end

  def rich_man?
    false
  end

private

  def method_missing(method, *args)
    nil
  end

  def toil_in_anatevka
    if self.rich_man?
      throw :miracle_of_miracles,
            'Emigrated to the Cayman Islands'
    end
  end

end

poor_tevye = Tevye.new
result = catch(:miracle_of_miracles) do
  poor_tevye.live
end

What is inevitable result of toiling in Anatevka? Nothing too surprising: it is being a :pauper.

But let’s change Tevye’s fortunes a bit and reward him richly for his labor. Take a look.

rich_tevye = Tevye.new
def rich_tevye.rich_man?
  true
end
result = catch(:miracle_of_miracles) do
  rich_tevye.live
end

What comes of Tevye’s hard work if deus ex machina descends upon the village? Tevye gets to retire to the western Caribbean, and his wife, Golde, grows a proper double chin.

Just as an unrescued error causes the Ruby process to crash, so a throw without a matching catch will raise an error (which in turn you can rescue).

It’s not just semantics

You cannot simply throw and catch wherever you would otherwise raise and rescue. There are differences beyond that the latter is for representing exception conditions and the former for other kinds of stack popping.

To begin with, (as you would expect from your experiences with other languages) rescue NameError rescues errors of type NameError as well as subclasses such as NoMethodError. By analogy you might presume that catch Entertainer would handle all these throw statements:

• throw Entertainer • throw Entertainer.new • throw BroadwayStar • throw BroadwayStar.new

You would be wrong: catch Entertainer handles only throw Entertainer.

Another important difference is that throw can take two arguments, as demonstrated in the Tevye class above. The second argument to throw is what Ruby returns from the matching catch statement (also demonstrated above).

A further distinction is the fact that rescue can appear more than once in a begin-end or def-end pair. The first matching rescue in a sequence wins.

begin
  do_something_error_prone
rescue AParticularKindOfError
  # Insert heroism here.
rescue Exception
  write_to_error_log
  raise
end

By contrast, catch is a block construct. Multiple catch statements must be nested in order to appear together.

catch :foo do
  catch :bar do
    do_something_that_can_throw_foo_or_bar
  end
end

Prior to Ruby v1.9, only symbols (for example, :pauper) could be thrown and caught. This restriction was removed, so now you can throw any Ruby object and catch it successfully.

The bottom line

When should you use this language feature? Not very often. As you probably noticed, my contrived example about peasant life in Tsarist Russia is a questionable application of the throw-catch pattern. There are two plausible examples in the Pickaxe book (http://ruby-doc.org/docs/ProgrammingRuby/html/tut_exceptions.html#S4). Fundamentally, throw-catch is a way to escape from the bowels of multiple layers of control flow without abusing raise-rescue for this purpose.

But if you have so many layers of flow control that you’re tempted to throw in order to get out of them quickly, consider trying to eliminate the layers first.

How to Make Your HTML Website Faster

1. Split Content

The chief purpose of responsive web design is to enhance web browsing experience on mobile devices without compromising on main contents of a website. But research findings reveal that around 80% people aren’t satisfied with mobile browsing experience, and slow performance of websites is a major reason of this disappointment. Clearly, responsive sites are facing performance issues which is chiefly because of too many HTML elements, excess loading time and extra content.

For seamless mobile browsing experience, it is very important to optimize websites so that their performance is accelerated. This is even more important for businesses because slow websites can have negative impact on marketing and promotion. Here are four actionable tips that will make responsive sites faster.

How to Make Your HTML Website Faster

A desktop website can afford to have many contents in a single webpage. But when responsive designing is used in it, then it is better to split its contents in separate web pages. This reduces HTML’s payload and avoids downloading of any associated dependencies. For example, when any product page of an ecommerce site is viewed on smaller screens, then the main product page should contain only generic information.

Keep separate web pages for ‘customer reviews’ and ‘new offers’ but, provide links for these pages within the main product page. Here, a single product page (for desktop website) splits into three different pages (when it becomes responsive). This will reduce HTTP requests as well as HTML size and will increase the site’s speed.

2. Optimize Images

High resolution images are great for desktop websites, but they are responsible for slowing down the performance of a responsive site. Images take the largest amount of KBs which is why websites with too many images tend to underperform on a mobile browser. Using CSS3 styling features like rounded corners and gradients fills will cut down HTTP requests and increase a site’s speed.

However, for images, one of the best techniques is using Adaptive Images. This software detects the screen size of a visitor’s device, and it automatically creates, caches and delivers an appropriate re-scaled version of embedded HTML images. There’s no need for mark-up changes.

3. Reduce JavaScript

JavaScript has significant effect on page speed. If your page has complex scripts, then mobile visitors of your website may have to spend few seconds in staring at blank page. This is because JavaScript increases a site’s loading time on tablet or smartphone. So reducing JavaScript can help your responsive site to perform better.

As a web designer, you must rethink using JavaScript and try to use it only when absolutely necessary. The best results can be achieved by placing scripts at the bottom of HTML. This prevents unnecessary blocking. However, if a script is absolutely necessary, then keep it inline and keep it small.

4. Use Conditional Loading

This is a very important and popular technique to enhance performance of responsive sites. Conditional loading prevents users to download site contents that make it slow. It can stop different kinds of content from downloading like maps, widgets and images. While implementing conditional loading, designers or developers should thoroughly test the site.

This should be done during optimization stages because it is easier to pick out flaws during that time. Further, it is easier to make corrections during optimization rather than doing so at the very end.

enter image description here

The Bottomline

Mobile users expect the same speed response while mobile browsing that they find in desktops and this expectation can be met only when responsive sites take minimum time to load. The above mentioned techniques can enhance speed of responsive sites by eliminating unnecessary contents and scripts. However, web designers must try to incorporate speed enhancing techniques during the build process of a website, and not as an afterthought.

Config Rails 3 with dalli(memcahed)

Config Rails 3 with dalli(memcahed)

To install dalli in your Rails 3 app, simply add the following to your Gemfile and run bundle install.

# Gemfile
gem "dalli", "~> 2.6.4",   :platforms => :ruby

To setup Rails to use the Dalli client for production, add the following to config/environments/production.rb:

# config/environments/production.rb
# Enable dalli (memcached) caching
config.cache_store = :dalli_store
config.action_controller.perform_caching = true

If you want to cache in any of your other environments, just add the same snippet to your environment of choice.

Dalli can also be used as rack middleware for session caching. To setup session caching with dalli, edit your config/initializers/session_store.rb to contain the following:

# config/initializers/session_store.rb
require 'action_dispatch/middleware/session/dalli_store'

Rails.application.config.session_store :dalli_store,
  :memcache_server => '127.0.0.1:11211',
  :namespace => 'sessions',
  :key => '_yourappname_session',
  :expire_after => 30.minutes

Now, whenever you use Rails’ built in caching or session storage, dalli will automatically interface with memcached.

# config/dalli.yml
# https://github.com/mperham/dalli#configuration
# parse the dalli.yml
# dalli_config = YAML.load_file(Rails.root.join('config/dalli.yml'))[Rails.env]
# memcached_hosts = dalli_config['servers']
# pass the servers to dalli setup
# config.cache_store = :dalli_store, *memcached_hosts, dalli_config["options"]
defaults: &defaults
  expires_in: 7 * 24 * 3600
  compress: true

development:
  options:
    <<: *defaults
    namespace: gxservice_development
  servers: 127.0.0.1:11211

test:
  options:
    <<: *defaults
    namespace: gxservice_test
  servers: 127.0.0.1:11211


staging:
  options:
    <<: *defaults
    namespace: gxservice_staging
  servers:
    - 127.0.0.1:11211

production:
  options:
    <<: *defaults
    namespace: gxservice_production
  servers:
    - cache01.mysite.org:11211
    - cache02.mysite.org:11211
    - cache03.mysite.org:11211

setup Rails use dalli with yaml file

# config/environments/production.rb
if defined?(Dalli)
  dalli_config = YAML.load_file(Rails.root.join('config/dalli.yml'))[Rails.env]
  memcached_hosts = dalli_config['servers']
  config.cache_store = :dalli_store, *memcached_hosts, dalli_config["options"]
end

migrate wordpress to heroku

WordPress Heroku

This project is a template for installing and running WordPress on Heroku. The repository comes bundled with PostgreSQL for WordPress and WP Read-Only.

Installation

Clone the repository from Github

$ git clone git://github.com/mhoofman/wordpress-heroku.git

With the Heroku gem, create your app

$ cd wordpress-heroku
$ heroku create
Creating strange-turtle-1234... done, stack is cedar
http://strange-turtle-1234.herokuapp.com/ | git@heroku.com:strange-turtle-1234.git
Git remote heroku added

Add a database to your app

$ heroku addons:add heroku-postgresql:dev
Adding heroku-postgresql:dev to strange-turtle-1234... done, v2 (free)
Attached as HEROKU_POSTGRESQL_COLOR
Database has been created and is available
Use `heroku addons:docs heroku-postgresql:dev` to view documentation

Promote the database (replace COLOR with the color name from the above output)

$ heroku pg:promote HEROKU_POSTGRESQL_COLOR
Promoting HEROKU_POSTGRESQL_COLOR to DATABASE_URL... done

Create a new branch for any configuration/setup changes needed

$ git checkout -b production

Copy the wp-config.php

$ cp wp-config-sample.php wp-config.php

Update unique keys and salts in wp-config.php on lines 48-55. WordPress can provide random values here.

define('AUTH_KEY',         'put your unique phrase here');
define('SECURE_AUTH_KEY',  'put your unique phrase here');
define('LOGGED_IN_KEY',    'put your unique phrase here');
define('NONCE_KEY',        'put your unique phrase here');
define('AUTH_SALT',        'put your unique phrase here');
define('SECURE_AUTH_SALT', 'put your unique phrase here');
define('LOGGED_IN_SALT',   'put your unique phrase here');
define('NONCE_SALT',       'put your unique phrase here');

Clear .gitignore and commit wp-config.php

$ >.gitignore
$ git add .
$ git commit -m "Initial WordPress commit"

Deploy to Heroku

$ git push heroku production:master
-----> Heroku receiving push
-----> PHP app detected
-----> Bundling Apache v2.2.22
-----> Bundling PHP v5.3.10
-----> Discovering process types
       Procfile declares types -> (none)
       Default types for PHP   -> web
-----> Compiled slug size is 13.8MB
-----> Launcing... done, v5
       http://strange-turtle-1234.herokuapp.com deployed to Heroku

To git@heroku:strange-turtle-1234.git
  * [new branch]    production -> master

After deployment WordPress has a few more steps to setup and thats it!

Usage

Because a file cannot be written to Heroku’s file system, updating and installing plugins or themes should be done locally and then pushed to Heroku.

Updating

Updating your WordPress version is just a matter of merging the updates into the branch created from the installation.

$ git pull # Get the latest

Using the same branch name from our installation:

$ git checkout production
$ git merge master # Merge latest
$ git push heroku production:master

WordPress needs to update the database. After push, navigate to:

http://your-app-url.herokuapp.com/wp-admin

WordPress will prompt for updating the database. After that you’ll be good to go.

Wiki

tips

heroku apps:rename jhjguxin --app evening-retreat-8008

plugin use by francis

├── plugins
│   ├── akismet
│   ├── google-analytics-for-wordpress
│   ├── hello.php
│   ├── index.php
│   ├── wordpress-importer
│   ├── wp-markdown
│   ├── wpro
│   └── xml-sitemap-feed
├── themes
│   ├── index.php
│   ├── twentyeleven
│   ├── twentythirteen
│   └── twentytwelve

tips:
plugin google-sitemap-generator need create ‘sitemap.xml’, ‘sitemap.xml.gz’ and permit CHMOD 666, can not do this on heroku, so use ‘xml-sitemap-feed’ instead it.

how to import from old wordpress

https://github.com/mhoofman/wordpress-heroku/issues/29

Same issue. Basically, since moving data from MySQL to PostgresSQL is such a huge pain, there is no way to transition from a classic wordpress install to wordpress since the importer is broken.

UPDATE: SOLVED!

To fix this issue: 1. Add the file you wish to import into your /wp-content/uploads/ directory structure and use the current year/month like “/wp-content/uploads/2013/01/blogname.wordpress.2013-01-10.xml_.txt (make sure to change the file extension) this is where wordpress would have uploaded the file). 2. Commit and push this new file to Heroku. 3. Disable the WordPress Read Only(WPRO) plugin. 4. Run the import like normal. 5. Re-enable the WPRO plugin

when throw ‘Error: The uploaded file exceeds the upload_max_filesize directive in php.ini

########
php_value upload_max_filesize 50M
php_value post_max_size 50M
php_value max_execution_time 500
php_value max_input_time 500
#########

Firing and being fired –by vibhu norby

source form Firing and being fired

I remember every detail from the day I was fired from my first post-college job. My manager pulled me into a meeting with the engineering director at the end of the day: “We have some tough news for you. We are letting you go. From the start, it wasn’t a good cultural fit. It’s not that you aren’t a good programmer. It’s just that this is not the right place for you. I’m sorry.”

In 30 seconds, all of my fears of failure and not being good enough to be a software engineer in Silicon Valley had come rushing back. And all of the work I had done in the past 7 months, the unfinished projects, the users I had connected with, the friendships with my colleagues – it all seemed wasted.

I wasn’t allowed to go back into the office and I fought tears as I was escorted out of the building. I sat in the parking lot for a long time and considered going somewhere else, doing something easier, working for a bigger company where there would be less pressure.

I remember working on my résumé that night and thinking that it was going to kill my chances of finding a new job if potential employers found out that I had been fired. Like many people who find themselves in my position, I mentally decided to answer the question of why I was only there for 7 months, if it arose, by saying that I had quit, or maybe that I had been consulting. Or the decidedly more ambiguous “I left.”

Two days later, I was in a garage with three exceptional people working at a new company on a problem I really believed needed solving. It was a company I had my eye on for a while. There, I was given the tools, the access to programming mentorship that I needed, and the inspiration to excel at my job from the top. I had managers that gave me the freedom to be creative, but also the boundaries to be successful. Our company was eventually acquired and it turned out that being fired was by far the best thing that could have ever happened to me.

It wasn’t until I had to fire my first person that I understood how success in your job is an indicator light that you’re on the right path, and not the other way around. People don’t succeed long term at their job because they’re just so good at what they do that nothing else matters. If they’re at the right company, they have the right managers, or they love what they’re doing, they’ll succeed no matter how good they were when they started. Success at anything feeds on itself and delivers continuous fruits.

I remember that day I fired my first employee clearly as well. I took the employee out for a walk with termination documents and a severance check, as well as a page of prepared notes on why I were letting the employee go. I explained to that person that it wasn’t a good fit from the start and that their performance wasn’t where we wanted, but that it had nothing to do with their capabilities as an engineer. It just wasn’t working.

Everything I told that engineer was true exactly as stated. I knew deep inside that everything the manager who fired me told me was true as well, despite my feeling at the time that I had done nothing wrong. In my first job, I worked on an ASP.NET site when I really wanted to work with open source programming languages and frameworks. In my first job, I was given rigid tasks to complete, and the many things that I came up with on my own rarely made it into production. In my first job, I worked for a game company when at the time I was interested in communication tools. Looking back, the only reason I didn’t leave on my own was that I didn’t have the self-awareness or courage to take my future into my own hands. So my job performance did the speaking for me.

Many software teams are really small, especially at startups. It’s crazy to think that every new employee is going to fit in with a small team when there is nowhere to hide like in a big company. A lack of passion for a company’s mission or product, even at a level that the employee is unable to verbalize or recognize, will show up in all sorts of ways. When you’re in a job that you’re not a good fit for, you’ll start feeling things like “nobody listens to me,” or you’ll notice that your manager doesn’t seem to appreciate your work or your work ethic. Sometimes your manager seems to be acting weird around you. Sometimes your manager asks you to change your attitude, and you do change, but it’s like they can’t see it. Those feelings are not necessarily real in the sense that your superiors are intentionally not listening to you or not appreciating your work or being weird. I believe that’s your mind signaling to you and interpreting situations with a lens for your wellbeing, basically telling you that you would be better off somewhere else. And indeed, you will be.

We’ve let go of half a dozen employees and kept only as many, and in every case where we let someone go, that person has gone on to work on things that they are more passionate about and where they have excelled and been happier at. When you let go of someone that’s not fitting in, the remaining team benefits from increased cohesiveness and self-worth. Having been on both sides of the equation now, I really believe that small companies do not fire people enough and that startup employees do not leave often enough. No interviewing process is good enough to completely understand a prospective employee’s deep inner passion for your company’s mission alongside evaluating their actual skills. It’s too easy to fake either one of those.

In hindsight, I wish the first company I worked for had fired me at the first sign of a lack of fit many months before. Sometimes you can detect a cultural mismatch within the first two months, sometimes in the first two weeks, and perhaps even on the first real day of work. It can be so easy to justify keeping the wrong employee on staff that a decision to fire an employee can drag on unresolved for months or years. However, both the employee and the company would almost always be better off if the separation happened immediately.

Posted in Uncategorized

通过show status 来优化MySQL数据库 from lxneng

1\. 查看MySQL服务器配置信息

mysql> show variables;

2\. 查看MySQL服务器运行的各种状态值

mysql> show global status;

3\. 慢查询

mysql> show variables like '%slow%';
+------------------+-------+
| Variable_name    | Value |
+------------------+-------+
| log_slow_queries | OFF   |
| slow_launch_time | 2     |
+------------------+-------+
mysql> show global status like '%slow%';
+---------------------+-------+
| Variable_name       | Value |
+---------------------+-------+
| Slow_launch_threads | 0     |
| Slow_queries        | 279   |
+---------------------+-------+

配置中关闭了记录慢查询(最好是打开,方便优化),超过2秒即为慢查询,一共有279条慢查询

4\. 连接数

mysql> show variables like 'max_connections';
+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| max_connections | 500   |
+-----------------+-------+

mysql> show global status like 'max_used_connections';
+----------------------+-------+
| Variable_name        | Value |
+----------------------+-------+
| Max_used_connections | 498   |
+----------------------+-------+

设置的最大连接数是500,而响应的连接数是498

max_used_connections / max_connections * 100% = 99.6% (理想值 ≈ 85%)

5\. key_buffer_size

key_buffer_size是对MyISAM表性能影响最大的一个参数, 不过数据库中多为Innodb

mysql> show variables like 'key_buffer_size';
+-----------------+----------+
| Variable_name   | Value    |
+-----------------+----------+
| key_buffer_size | 67108864 |
+-----------------+----------+

mysql> show global status like 'key_read%';
+-------------------+----------+
| Variable_name     | Value    |
+-------------------+----------+
| Key_read_requests | 25629497 |
| Key_reads         | 66071    |
+-------------------+----------+

一共有25629497个索引读取请求,有66071个请求在内存中没有找到直接从硬盘读取索引,计算索引未命中缓存的概率:

key_cache_miss_rate = Key_reads / Key_read_requests * 100% =0.27%

需要适当加大key_buffer_size

mysql> show global status like 'key_blocks_u%';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| Key_blocks_unused | 10285 |
| Key_blocks_used   | 47705 |
+-------------------+-------+

Key_blocks_unused表示未使用的缓存簇(blocks)数,Key_blocks_used表示曾经用到的最大的blocks
Key_blocks_used / (Key_blocks_unused + Key_blocks_used) * 100% ≈ 18% (理想值 ≈ 80%)

6\. 临时表

mysql> show global status like 'created_tmp%';
+-------------------------+---------+
| Variable_name           | Value   |
+-------------------------+---------+
| Created_tmp_disk_tables | 4184337 |
| Created_tmp_files       | 4124    |
| Created_tmp_tables      | 4215028 |
+-------------------------+---------+

每次创建临时表,Created_tmp_tables增加,如果是在磁盘上创建临时表,Created_tmp_disk_tables也增加,Created_tmp_files表示MySQL服务创建的临时文件文件数:

Created_tmp_disk_tables / Created_tmp_tables * 100% = 99% (理想值<= 25%)

mysql> show variables where Variable_name in ('tmp_table_size', 'max_heap_table_size');
+---------------------+-----------+
| Variable_name       | Value     |
+---------------------+-----------+
| max_heap_table_size | 134217728 |
| tmp_table_size      | 134217728 |
+---------------------+-----------+

需要增加tmp_table_size

7\. open table 的情况

mysql> show global status like 'open%tables%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Open_tables   | 1024  |
| Opened_tables | 1465  |
+---------------+-------+

Open_tables 表示打开表的数量,Opened_tables表示打开过的表数量,如果Opened_tables数量过大,说明配置中 table_cache(5.1.3之后这个值叫做table_open_cache)值可能太小,我们查询一下服务器table_cache值

mysql> show variables like 'table_cache';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| table_cache   | 1024  |
+---------------+-------+

Open_tables / Opened_tables * 100% = 69% 理想值 (>= 85%)
Open_tables / table_cache * 100% = 100% 理想值 (<= 95%)

8\. 进程使用情况

mysql> show global status like 'Thread%';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| Threads_cached    | 31    |
| Threads_connected | 239   |
| Threads_created   | 2914  |
| Threads_running   | 4     |
+-------------------+-------+

如果我们在MySQL服务器配置文件中设置了thread_cache_size,当客户端断开之后,服务器处理此客户的线程将会缓存起来以响应下一个客户而不是销毁(前提是缓存数未达上限)。Threads_created表示创建过的线程数,如果发现Threads_created值过大的话,表明 MySQL服务器一直在创建线程,这也是比较耗资源,可以适当增加配置文件中thread_cache_size值,查询服务器 thread_cache_size配置:

mysql> show variables like 'thread_cache_size';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| thread_cache_size | 32    |
+-------------------+-------+

9\. 查询缓存(query cache)

mysql> show global status like 'qcache%';
+-------------------------+----------+
| Variable_name           | Value    |
+-------------------------+----------+
| Qcache_free_blocks      | 2226     |
| Qcache_free_memory      | 10794944 |
| Qcache_hits             | 5385458  |
| Qcache_inserts          | 1806301  |
| Qcache_lowmem_prunes    | 433101   |
| Qcache_not_cached       | 4429464  |
| Qcache_queries_in_cache | 7168     |
| Qcache_total_blocks     | 16820    |
+-------------------------+----------+

Qcache_free_blocks:缓存中相邻内存块的个数。数目大说明可能有碎片。FLUSH QUERY CACHE会对缓存中的碎片进行整理,从而得到一个空闲块。
Qcache_free_memory:缓存中的空闲内存。
Qcache_hits:每次查询在缓存中命中时就增大
Qcache_inserts:每次插入一个查询时就增大。命中次数除以插入次数就是不中比率。
Qcache_lowmem_prunes:缓存出现内存不足并且必须要进行清理以便为更多查询提供空间的次数。这个数字最好长时间来看;如果这个数字在不断增长,就表示可能碎片非常严重,或者内存很少。(上面的 free_blocks和free_memory可以告诉您属于哪种情况)
Qcache_not_cached:不适合进行缓存的查询的数量,通常是由于这些查询不是 SELECT 语句或者用了now()之类的函数。
Qcache_queries_in_cache:当前缓存的查询(和响应)的数量。
Qcache_total_blocks:缓存中块的数量。

我们再查询一下服务器关于query_cache的配置:

mysql> show variables like 'query_cache%';
+------------------------------+----------+
| Variable_name                | Value    |
+------------------------------+----------+
| query_cache_limit            | 33554432 |
| query_cache_min_res_unit     | 4096     |
| query_cache_size             | 33554432 |
| query_cache_type             | ON       |
| query_cache_wlock_invalidate | OFF      |
+------------------------------+----------+

各字段的解释:

query_cache_limit:超过此大小的查询将不缓存
query_cache_min_res_unit:缓存块的最小大小
query_cache_size:查询缓存大小
query_cache_type:缓存类型,决定缓存什么样的查询,示例中表示不缓存 select sql_no_cache 查询
query_cache_wlock_invalidate:当有其他客户端正在对MyISAM表进行写操作时,如果查询在query cache中,是否返回cache结果还是等写操作完成再读表获取结果。

query_cache_min_res_unit 的配置是一柄”双刃剑”,默认是4KB,设置值大对大数据查询有好处,但如果你的查询都是小数据查询,就容易造成内存碎片和浪费。

查询缓存碎片率 = Qcache_free_blocks / Qcache_total_blocks * 100%

如果查询缓存碎片率超过20%,可以用FLUSH QUERY CACHE整理缓存碎片,或者试试减小query_cache_min_res_unit,如果你的查询都是小数据量的话。

查询缓存利用率 = (query_cache_size – Qcache_free_memory) / query_cache_size * 100%

查询缓存利用率在25%以下的话说明query_cache_size设置的过大,可适当减小;查询缓存利用率在80%以上而且Qcache_lowmem_prunes > 50的话说明query_cache_size可能有点小,要不就是碎片太多。

查询缓存命中率 = (Qcache_hits – Qcache_inserts) / Qcache_hits * 100%

示例服务器 查询缓存碎片率 = 20.46%查询缓存利用率 = 62.26%查询缓存命中率 = 1.94%,命中率很差,可能写操作比较频繁吧,而且可能有些碎片。

10\. 排序使用情况

mysql> show global status like 'sort%';
+-------------------+----------+
| Variable_name     | Value    |
+-------------------+----------+
| Sort_merge_passes | 2136     |
| Sort_range        | 81888    |
| Sort_rows         | 35918141 |
| Sort_scan         | 55269    |
+-------------------+----------+

Sort_merge_passes 包括两步。MySQL 首先会尝试在内存中做排序,使用的内存大小由系统变量 Sort_buffer_size 决定,如果它的大小不够把所有的记录都读到内存中,MySQL 就会把每次在内存中排序的结果存到临时文件中,等 MySQL 找到所有记录之后,再把临时文件中的记录做一次排序。这再次排序就会增加 Sort_merge_passes。实际上,MySQL 会用另一个临时文件来存再次排序的结果,所以通常会看到 Sort_merge_passes 增加的数值是建临时文件数的两倍。因为用到了临时文件,所以速度可能会比较慢,增加 Sort_buffer_size 会减少 Sort_merge_passes 和 创建临时文件的次数。但盲目的增加 Sort_buffer_size 并不一定能提高速度,见 How fast can you sort data with MySQL?(引自http://qroom.blogspot.com/2007/09/mysql-select-sort.html)

另外,增加read_rnd_buffer_size(3.2.3是record_rnd_buffer_size)的值对排序的操作也有一点的好处,参见:http://www.mysqlperformanceblog.com/2007/07/24/what-exactly-is- read_rnd_buffer_size/

11\. 文件打开数(open_files)

mysql> show global status like 'open_files';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Open_files    | 821   |
+---------------+-------+

mysql> show variables like 'open_files_limit';
+------------------+-------+
| Variable_name    | Value |
+------------------+-------+
| open_files_limit | 65535 |
+------------------+-------+

比较合适的设置:Open_files / open_files_limit * 100% <= 75%

12\. 表锁情况

mysql> show global status like 'table_locks%';
+-----------------------+---------+
| Variable_name         | Value   |
+-----------------------+---------+
| Table_locks_immediate | 4257944 |
| Table_locks_waited    | 25182   |
+-----------------------+---------+

Table_locks_immediate 表示立即释放表锁数,Table_locks_waited表示需要等待的表锁数,如果 Table_locks_immediate / Table_locks_waited > 5000,最好采用InnoDB引擎,因为InnoDB是行锁而MyISAM是表锁,对于高并发写入的应用InnoDB效果会好些.

13\. 表扫描情况

mysql> show global status like 'handler_read%';
+-----------------------+-----------+
| Variable_name         | Value     |
+-----------------------+-----------+
| Handler_read_first    | 108763    |
| Handler_read_key      | 92813521  |
| Handler_read_next     | 486650793 |
| Handler_read_prev     | 688726    |
| Handler_read_rnd      | 9321362   |
| Handler_read_rnd_next | 153086384 |
+-----------------------+-----------+

各字段解释参见http://hi.baidu.com/thinkinginlamp/blog/item/31690cd7c4bc5cdaa144df9c.html,调出服务器完成的查询请求次数:

mysql> show global status like 'com_select';
+---------------+---------+
| Variable_name | Value   |
+---------------+---------+
| Com_select    | 2693147 |
+---------------+---------+

计算表扫描率:

表扫描率 = Handler_read_rnd_next / Com_select

如果表扫描率超过4000,说明进行了太多表扫描,很有可能索引没有建好,增加read_buffer_size值会有一些好处,但最好不要超过8MB。

Posted in Uncategorized

HOW TO CHECK AND FIND YOUR LINUX OS VERSION?

Linux and Unix operating systems comes in a wide range of flavors often bundled as different distributions by different vendors. Every one of these distribution also comes with an often pre-defined and latest version of the Linux kernel.
Sometimes you need to know the exact name and version of your operating system, machine as well as the kernel, be it to install the correct version of a software, find if a hardware is compatible or be it to upgrade your OS itself. There are several ways to check your operating system and linux kernel versions. As each distribution (or distros) are slightly different some of the commands might work in some distros while some maynot.

uname

uname is the linux command which prints out the name, versions and other details of the machine and kernel running on the machine. It is basically short for Unix Name. This is usually part of the core-utils package and should be available on almost all distros. There are several options available to print out just the kernel details or just the machine information.
To print out just the kernel information, use the -srv option. It prints out all the available kernel information.

bash$ uname -srv

Output:

Linux 3.3.0-gentoo #2 SMP PREEMPT Wed Mar 21 02:07:10 CDT 2012

The first part prints out the kernel name, which is Linux in the above example. The second part is the kernel release version, which is 3.3.0-gentoo. The rest of it is a more detailed kernel information like the compilation date and config.
To print out the machine information, use the -mnipo options.

bash$ uname -mnipo

Output:

machinename i686 Intel(R) Core(TM)2 Duo CPU E6850 @ 3.00GHz GenuineIntel GNU/Linux

machinename is the name of the machine, while the rest is the processor architecture, processor type, version, speed and operating system information.

You can also use the -a option which prints out all the available information about the kernel and the machine.

etc release and version files

Some distributions ships with a separate set of files which specify the release and versions of the distro. These files are usually in the /etc folder with either the word release or version in them or two different files specifying both. You can view these files using any text editor or the cat command.

bash$ cat /etc/*-release*

Output:

Gentoo Base System release 2.1
 DISTRIB_ID="Gentoo"

Also try,

bash$ cat /etc/*-version*

and

bash$ cat /etc/issue

lsb_release

lsb_release (Linux Standard Base Release) is another command which prints out useful information about the distribution. The command has several options to print out specific information, but the -a or the –all option prints out all the information.

bash$ lsb_release -a

Output:

LSB Version: n/a
 Distributor ID: Gentoo
 Description: Gentoo Base System release 2.1
 Release: 2.1
 Codename: n/a

proc version

Another option you have is to check the proc version. You can do so by using the cat command to print out the contents of the
/proc/version file.

bash$ cat /proc/version

Output:

Linux version 3.3.0-gentoo (root@machinename) (gcc version 4.5.3 (Gentoo 4.5.3-r2 p1.1, pie-0.4.7) ) #2 SMP PREEMPT Wed Mar 21 02:07:10 CDT 2012

This prints out a complete and detailed list of information about your kernel, processor, machine and operating system.

Posted in Uncategorized

vagreant tutorial

vagreant tutorial

this tutorial base on vagrant-tutorial and suit for ~> 1.2 vagrant, thanks author @gogojimmy

what is vagreant

Vagrant is a tool for building and distributing virtualized development environments.

install vagreant

Vagrant 背後用的是 Virtual Box 作為虛擬機器, Vagrant 只是一個讓你可以方面做設定來開你想要的虛擬機器的方便工具, 所以你必須先安裝 Vagrant 和 Virtual Box, Virtual Box 你可以在Virtual Box 官網下載適合你平台的版本,而 Vagrant 你可以在 Vagrant 官網下載打包好的版本,或是如果你跟我一樣是個玉樹臨風的 Rubist, 你可以打開我們最愛的小黑視窗輸入

$ gem install vagrant # not recomment

Vagrant 1.1後已經不支援使用Gem來安裝了 http://mitchellh.com/abandoning-rubygems

$# wget http://files.vagrantup.com/packages/7e400d00a3c5a0fdf2809c8b5001a035415a607b/vagrant_1.2.2_i686.deb -O ~/Downloads/vagrant_1.2.2_i686.deb && sudo dpkg -i ~/Downloads/vagrant_1.2.2_i686.deb
$ wget http://files.vagrantup.com/packages/7e400d00a3c5a0fdf2809c8b5001a035415a607b/vagrant_1.2.2_i686.deb -P ~/Downloads/ && sudo dpkg -i ~/Downloads/vagrant_1.2.2_i686.deb

開始使用Vagrant 新增作業系統

當你已經安裝好 Virtual Box 以及 Vagrant 後,你要開始思考你想要在你的VM上使用什麼作業系統,一個打包好的作業系統環境在 Vagrant 稱之為 Box,也就是說每個 Box 都是一個打包好的作業系統環境,當然網路上什麼都有,你不用自己去找作業系統,vagrantbox.es 上面就有許多大家熟知且已經打包好的作業系統,你只需要下載就可以了,為你的Vagrant增加一個 Box 很簡單

# Vagrant version 1.2.2 should use bellow
$ vagrant box add {你想要的Box名稱} {下載網址}
# vagrant box add ubuntu-13-04 ~/raring-server-cloudimg-amd64-vagrant-disk.box
# vagrant box add ubuntu-13-04 raring-server-cloudimg-amd64-vagrant-disk.box
$ vagrant box add ubuntu-13-04  http://cloud-images.ubuntu.com/raring/current/raring-server-cloudimg-vagrant-amd64-disk1.box

# not work for francis
$ vagrant box add ubuntu-13-04 http://cloud-images.ubuntu.com/raring/current/raring-server-cloudimg-vagrant-amd64-disk1.box
vagrant box list
#vagrant box remove ubuntu-12-10 virtualbox

# box add centos-6_4-x86_64 CentOS-6.4-x86_64.box
# box add centos-6_4-x86_64 CentOS-6.4-x86_64.box

Vagrant 就會開始下載這個 Box,你可以用vagrant box list這個指令看到你所擁有的所有 Box,想像就是你的書架上多了一片 Ubuntu 12.10 的安裝光碟,以後要安裝機器就是用這的安裝就可以了,有了Box以後,我們要產生一個設定檔來設定我們的虛擬機器,這個檔案可以透過指令 vagrant init Box名稱來產生,你可以在你的專案中或是另外開個練習用的資料夾輸入,這時候你的資料夾終究會有一個名稱為vagrantfile的檔案,這個檔案就是所有魔法的開始

$ mkdir vagrant-boxes && cd vagrant-boxes
vagrant init ubuntu-13-04
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.

讓VM動起來

我們晚一點再提設定檔的部份,讓我們先把VM跑起來,要讓VM跑起來的指令是vagrant up

#vagrant destroy &&VAGRANT_LOG=DEBUG vagrant up
VAGRANT_LOG=DEBUG vagrant up
vagrant status
Current machine states:

default                  running (virtualbox)

The VM is running. To stop this VM, you can run `vagrant halt` to
shut it down forcefully, or you can run `vagrant suspend` to simply
suspend the virtual machine. In either case, to restart it again,
simply run `vagrant up`.

known issue https://github.com/mitchellh/vagrant/issues/455

sudo rm /etc/udev/rules.d/70-persistent-net.rules
sudo touch /etc/udev/rules.d/70-persistent-net.rules
sudo ufw allow 2222
sudo ufw disable

#Vagrantfile
Vagrant::Config.run do |config|
  config.vm.network :bridged, :bridge => "en0: Wi-Fi (AirPort)"
end

#http://docs-v1.vagrantup.com/v1/docs/config/ssh/port.html

將習慣的環境打包成Box

我的環境大概要裝15分鐘左右, 有人說一個程式設計師的人生花最多時間的一件事情就是等, 這個說法真是一點也不為過, 每次裝機都從環境重新開始 Build 也不是辦法,我們要讓人生有更多的時間去處理更多的事情, 所以我們可以把一個已經Build好的環境打包成一個我們自己的 Box, 以後我們只要直接使用這個打包好的版本就可以了, 因此讓我們幫自己的人生省點時間, 速速登出VM來打包這個 Box

$ vagrant package
[default] Attempting graceful shutdown of VM...
[default] Clearing any previously set forwarded ports...
[default] Creating temporary directory for export...
[default] Exporting VM...
[default] Compressing package to: /home/jhjguxin/vagrant-boxes/ubuntu-server-12042/package.box

vagrant package 這個指令會在你目前的資料夾下建立一個 package.box 的 Box 檔案, 這時候我們跟剛剛一樣把它加入到我們的Box List中, 以後我們就可以快速使用這個 Box 就好了! 除此之外, 可以自定 Box 的意義還有讓你的團隊都能用VM來擁有自己的 Staging 環境,例如在 Rails 專案中我們也可以建立一個 Vagrant 的設定檔來做一個給開發人員測試用的 Staging 環境, 這時候你就可以指定好你自定的機器設定, 確保每個開發人員都能擁有一樣的環境來進行開發.

$ vagrant box add francis-ubuntu-server-12042 package.box
$ vagrant box list

Vagrant 基本設定

設定VM的名稱及記憶體

config.vm.box = "francis-ubuntu-server-12042"

這告訴了 Vagrant 要去套用哪個 Box 作為環境, 也就是你一開始輸入 varant init Box名稱時所指定的Box, 如果沒有輸入Box名稱的話就會是預設的base, Virtual Box本身提供了VBoxManage這個command line tool讓你可以設定你的VM, 用 modifyvm 這個指令讓你可以設定VM的名稱及記憶體大小等等,這裡說的名稱指的是在Virtual Box中顯示的名稱,我們也可以在 VagrantFile 中進行設定,在你的 vagrantfile 中加入這行

config.vm.customize ["modifyvm", :id, "--name", "francis-ubuntu-server-12042", "--memory", "512"]

行設定檔意思就是呼叫 VBoxManage 的 modifyvm 的指令, 設定VM的名稱為 `francis-ubuntu-server-12042′ 而設定VM的記憶體大小為 512MB,你可以照這這種作法為你的 VM 設定好不同的設定.

設定 Hostname 以及 Port forward

# 設定 hostname 非常重要,有很多服務都仰賴著 hostname 來做為辨識,例如Puppet或是Chef,一般一些監控服務像是New Relic之類的也都是以 hostname 來做為辨識
config.vm.host_name = "gogojimmy-app"
# 把Host機器上8080 port傳來的東西forward到VM跑的 80 port的服務
config.vm.forward_port 80, 8080

設定網路橋接方式

Vagrant有兩種橋接方式是, 一種是host only, 意思是說在你電腦同個區網中的其他電腦是看不到你的VM的, 只有你一個人自 High, 另一種是Bridge, 當然就是說VM會跟你區網的router去要一組 IP, 區網中的其他電腦也都能看到他, 般來說因為開 VM 的情況都是自 High居多, 因此我們在設定上都是設定host only:

config.vm.network :hostonly, "33.33.13.10"

這邊將網路設定成 hostonly, 並且指定一組 IP 位址, IP 位址的設定會建議不要使用 192.168.. 的設定, 因為很有可能會跟你區網的IP衝突, 你可以改使用像是 33.33.. 的設定.

更改vagrantfile的設定後,記得要用vagrant reload的指令重開VM讓VM可以用新的設定檔跑起來

讓我們開始打造多機器環境

重頭戲來了, 前面的一切都是為了今天鋪陳, 現在我們要建立多個 VM 跑起來,並且讓他們互相溝通, 有人跑 Application, 有人跑DB, 有人跑Memcached, 這一切在 Vagrant 中非常簡單, 跟剛剛的設定都一樣, 你只需要指定好機器的角色就可以了, 讓我們再次打開我們的設定檔來設定一台APP Server加上一台DB Server:

# for v1 ~> 1.1
config.vm.define :app do |app_config|
    app_config.vm.customize ["modifyvm", :id, "--name", "app", "--memory", "512"]
    app_config.vm.box = "ubuntu-server-12042"
    app_config.vm.host_name = "app"
    app_config.vm.network :hostonly, "33.33.13.10"
end
config.vm.define :db do |db_config|
  db_config.vm.customize ["modifyvm", :id, "--name", "db", "--memory", "512"]
  db_config.vm.box = "ubuntu-server-12042"
  db_config.vm.host_name = "db"
  db_config.vm.network :hostonly, "33.33.13.11"
end

# for v2 ~> 1.2
config.vm.provision :shell, :inline => "echo Hello"
config.vm.define :app do |app_config|
  app_config.vm.provider "virtualbox" do |v|
    v.customize ["modifyvm", :id, "--name", "app", "--memory", "512"]
  end
  app_config.vm.box = "ubuntu-server-12042"
  app_config.vm.hostname = "app"
  app_config.vm.network :private_network, ip: "33.33.13.10"
end
config.vm.define :db do |db_config|
  db_config.vm.provider "virtualbox" do |v|
    v.customize ["modifyvm", :id, "--name", "db", "--memory", "512"]
  end
  db_config.vm.box = "ubuntu-server-12042"
  db_config.vm.hostname = "db"
  db_config.vm.network :private_network, ip: "33.33.13.11"
end

這邊的設定就像是剛剛在設定的部份教的一樣, 只是我們使用了 :app 以及:db 分別做了兩個 VM 的設定, 並且給予不同的 hostname 和IP, 設定好了以後再使用 vagrant up 將機器跑起來:

$ vagrant up
Bringing machine 'app' up with 'virtualbox' provider...
Bringing machine 'db' up with 'virtualbox' provider...
[app] Importing base box 'ubuntu-server-12042'...
[app] Matching MAC address for NAT networking...
[app] Setting the name of the VM...
[app] Clearing any previously set forwarded ports...
[app] Creating shared folders metadata...
[app] Clearing any previously set network interfaces...
[app] Preparing network interfaces based on configuration...
[app] Forwarding ports...
[app] -- 22 => 2222 (adapter 1)
[app] Running any VM customizations...
[app] Booting VM...
[app] Waiting for VM to boot. This can take a few minutes.
[app] VM booted and ready for use!
[app] Setting hostname...
[app] Configuring and enabling network interfaces...
[app] Mounting shared folders...
[app] -- /vagrant
[db] Importing base box 'ubuntu-server-12042'...
[db] Matching MAC address for NAT networking...
[db] Setting the name of the VM...
[db] Clearing any previously set forwarded ports...
[db] Fixed port collision for 22 => 2222. Now on port 2200.
[db] Creating shared folders metadata...
[db] Clearing any previously set network interfaces...
[db] Preparing network interfaces based on configuration...
[db] Forwarding ports...
[db] -- 22 => 2200 (adapter 1)
[db] Running any VM customizations...
[db] Booting VM...
[db] Waiting for VM to boot. This can take a few minutes.
[db] VM booted and ready for use!
[db] Setting hostname...
[db] Configuring and enabling network interfaces...
[db] Mounting shared folders...
[db] -- /vagrant

看到上面的訊息跑完後,你就可以跟剛剛一樣使用 ssh 連到VM裡,但這次不同的是你要加上你所指定的角色告訴你要連線的機器是哪一台:

$vagrant ssh app vagrant ssh app
Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.5.0-23-generic x86_64)

 * Documentation:  https://help.ubuntu.com/
Last login: Wed Apr 10 14:03:49 2013 from 10.0.2.2
vagrant@app:~$

是不是很酷!!再來我們來驗證一下 VM 之間的連線,讓我們使用 ssh 登入 db 的機器,然後在 db 的機器上使用 ssh 來連線到 app 的機器(預設密碼就是vagrant):

看到了嗎, VM之間的溝通也是沒有問題的! 你現在可以開始好好思考你偉大的Infrastructure, 讓你的程式跑在多機器的環境中, 如果你對於 Infrastructure 不熟悉, Amazon有提供了不少範例可以參考, 想像力就是你的超能力, 現在唯一侷限你的只會是你的電腦記憶體了, 不要開到跑不動都不會有事的, 今天就開始用Vagrant練習你的機器佈署吧!

Posted in Uncategorized

分享一些资料(侧重Linux) –by D瓜哥

以前没有攒资料的习惯,这一两年才养成了这个习惯。有次和大学老师聊天,和她说攒了不少资料,她就让我分享给她看。后来和她进一步沟通,觉得整理一下,分给学弟学妹看也许对他们更有帮助。为了这个目标,把资料重新梳理了一下,并且添加了一点我个人对学习方法的理解,分享出来,希望对大家有所帮助!

  本文章面向刚刚踏入挨踢行业的新手,高手可以直接飞过!当然,如果有好的资料,或者一些不同的看法,请留言交流!

学习方法论

享受学习!寻找学习中的乐子,享受这种乐子!强化学习中的成就感,提高自己对课程,对学习,对生活的自信心!在《只要一部计算机,就可以创造出无限的世界》中有更多论述。请自己查看。

亲自动手实践!就我的同学来看,大家最欠缺的是动手能力。很多人理论学的还行,但是一动手就完蛋!所以,一定要亲自动手实践!!另外,你可以把你实践的过程记录下来,发布到网上。说不定有很多菜鸟需要你的这些资料呢。
最好的学习方式就是把学习的东西记下来!当你试着把你所学的东西都记录下来时,你就会逼着自己去把不懂的东西搞清楚。另外,再你写的过程中,你要斟词酌句,你要谋篇布局,还锻炼你分析问题、整理问题的能力。无形中让你受益匪浅。所以,强烈推荐把自己学的东西写成博客发布出来!目前中Linux方面最好的论坛算是51CTO(D瓜哥个人认为)了。大家可以在上面注册个账号,开始写自己的博客!如果你写得好,会有很多人赞你!这可以获得很大的成就感,更会提高你的写作动力。如果你写的有问题,一些热心网友也会给你指出,纠正你的错误,你还是会获得进步。所以,无论从哪个方面讲,写博客都是百利而无一害的!所以,有事没事写写博客还是大有益处的!如果哪天你不小心混到VIP,那就真的牛逼了!当然,你所学的任何方面的东西都可以写出来发布出来,不限Linux。
广告一下:如果感觉这篇文章对您有用,而且方便的话,麻烦把我的网站“‘地瓜哥’博客网”加入到你的友情链接列表里!如果是推荐博客列表更好!哈哈

一定一定要学会使用Google!把使用Google查资料培养成一种习惯!有事没事用Google百度一下!这里给大家讲解一个这段时间发生的一个故事:15岁的美国学生Jack Andraka的长辈因胰脏腺癌过世,让他决定上网用谷歌和维基百科理解这噩病。他发现:初期胰脏癌很好医治,但后期几乎无望,所以早期诊断是最关键的。7个月后他发明了一种纳米试纸,能够迅速廉价精准检测胰腺癌生物标志。由此可见,互联网时代人人都可以成为科学家!相信瓜哥,没错的!也许有人会提到百度,D瓜哥看来如果你不是做假药的,百度真的就可以算了!珍爱生命,从远离百度开始!
本文发出来之后,一些网友说D瓜哥藐视百度等等。D瓜哥这里做个说明,D瓜哥不藐视百度;D瓜哥只是认为百度搜索结果中各种推广等太多了,严重影响了使用体验;尤其是对于挨踢方面的搜索结果不够精准,还要一页一页翻着找,太浪费时间!另外,我们可以想想,为什么很多人忍受这个Google断网、链接重置、网速慢等问题,继续坚持使用Google?当然,D瓜哥只是建议使用Google。至于如何选择,完全看自己!

一定要把英语学好!在挨踢行业中,英语几乎是个必须掌握的技能!学好英语,就可以打开一个更广阔的天地。能接触到最新的各种挨踢资料。全世界最好的挨踢问答网StackOverflow就是以英语为主,学好英语,你所有的挨踢疑难杂症,几乎都可以在这里找到。另外,里面还有好多经常的解答,也会让你受益匪浅!
也许大家会觉得英语学起来很费劲,看原版书更是头大!其实,并不像大家想象的那样!D瓜哥的英语也很烂,但是D瓜哥也读过几本英语原版书了!根据我的个人经验,刚拿到一本书确实很头大,但是只要硬着头皮看几章之后,找到感觉之后,读起来就会顺畅很多了!遇到不懂的,拿个词典(电子词典也行,推荐有道词典)随时查就行了。

  推荐大家自己搭建一个博客,可以自己开发,也可以使用Wordpress。注册个域名,配置DNS,你要懂网络知识;搭建博客,无论是自己开发,还是定制Wordpress,你都要懂PHP(当然,也可以是其他编程语言!不过,网络上PHP的虚拟主机最多,也便宜!JSP的虚拟主机少,而且贵!);如果再进行网站优化,那就需要懂更多的知识了,编程语言、互联网协议、数据库优化、Linux服务器管理、HTTP、缓存、JavaScript、CSS、DNS等等!可以把你自己所学的东西,都串起来,构成一个整体的、清晰的框架!

Linux资料

  1. 只要一部计算机,就可以创造出无限的世界,来个开胃菜,准备迎接下面的大餐!
  2. 程序猿技术练级攻略
  3. 应该知道的Linux技巧
  4. 命令行是你的至交好友
  5. LinuxCast
  6. 28个Unix/Linux的命令行神器
  7. 简明VIM教程
  8. 普通人的编辑利器——Vim
  9. Vimer的程序世界
  10. 手把手教你把Vim改装成一个IDE编程环境(图文)
  11. All commands,非常强大的Linux命令在线查询工具,谁用谁知道!!
  12. Linux Wiki,里面的指令讲解挺详细,排版也很好!
  13. AWK简明教程
  14. sed简明教程
  15. Unix目录结构的来历
  16. Unix传奇(上篇)
  17. Unix传奇(下篇)
  18. Unix 40年:昨天,今天和明天
  19. 理解inode
  20. KVM虚拟化原理与实践,最新颖的Linux虚拟机技术!
  21. agentzh 的 Nginx 教程,目前使用最广泛的Web服务器!

网络相关

  1. 互联网协议入门(一),从上到下,从下到上,透彻地分析互联网中的主要协议!
  2. 互联网协议入门(二)
  3. SSH原理与运用(一):远程登录
  4. SSH原理与运用(二):远程操作与端口转发
  5. 趣解HTTP状态码
  6. CDN技术介绍
  7. CDN技术实践

编程相关

  1. Web开发:我希望得到的编程学习路线图,这只是一个学习指导路线。大家可以把其中Ruby相关的,换成JSP、ASP.NET或者PHP等。另外,Ruby很好玩,感兴趣的可以学一学。(如果Java或者C#等没有学会的话,还是不要学了,Ruby的语法太灵活了,你会被搞蒙的。)
  2. 23种设计模式的形象比喻
  3. JDK里的设计模式
  4. 每个程序员都应该了解的内存知识
  5. Codecademy,在线交互式编程学习网站!很有趣!
  6. MySQL(微博),里面分享了很多干货
  7. 图解SQL的Join
  8. 字符编码笔记:ASCII,Unicode和UTF-8
  9. Redis 设计与实现,可能一时半会用不上,备用吧。

网页开发

  1. jQuery选择器大全,jQuery是任何一个搞网站开发人员,甚至移动应用开发人员必学JavaScript库!
  2. Bootstrap,Github上最火爆的项目。
  3. Bootstrap中文手册
  4. 50个必备的实用jQuery代码段
  5. jsFiddle,可以在网上调试JavaScript代码。还可以共享出来。
  6. HTML语义简析
  7. 优设,网页设计。
  8. imageCSS
  9. 浏览器的工作原理
  10. 浏览器是如何工作的?
  11. “分享一些D瓜哥攒的比较好的Web开发资料”,这里分享了更多的关于这方面的资料!

书籍推荐

  其实,读书也可以很有趣,比如下面的这些书。

  1. 鸟哥的Linux私房菜.基础学习篇
  2. 大话设计模式,这本书会告诉你,其实技术也很好看。
  3. 重来,静下心来,两个小时看完。
  4. 黑客与画家,阮一峰(下面有推荐他的博客)翻译的,书不错,推荐看看!
  5. 数学之美,让你切身体验一下计算机中是怎么玩数学的。
  6. 浪潮之巅,和上面的《数学之美》是同一个人,吴军博士,Google攻城师,前腾讯副总裁。
  7. 思考的乐趣,其实数学也很好玩!不信你看看这本书。另外,这本书的作者是中文系的!是不是突然感觉很牛逼啊?加油,你也可以很牛逼!
  8. JavaScript高级程序设计(第3版),学JavaScript只需要这一本书;
  9. 明朝那些事儿,很好看的历史书!这里只给出了第一本的链接,一共七本!大家也可以在网上找电子版的。这套书最早就是电子书!
  10. 牛奶可乐经济学,生活中很多有趣的现象都可以使用经济学的一些原理来解释。问题有趣,解释也很有意识!喜欢经济学的朋友可以看看这套书。这也是一套书,目前出到第三本了。
  11. Unix编程艺术,作者是Unix的开发人员,开源运动的推动者,主要讲解Unix/Linux的“野史”;当然,也讲解了很多Unix/Linux系统设计原理和设计思想。推荐!

挨踢趣闻

  1. 黑客的含义
  2. 那些年,MIT技术男们奇妙的黑客作品
  3. 淘宝技术发展
  4. [Tech Drafts](http://www.techdrafts.com/
  5. 士兵,间谍,服务器
  6. 给哥三十五次机会,哥就能猜中你的手机号

知名技术博客

  这里推荐的一般是个人博客,知名论坛等,不在此列。

  1. 地瓜哥,我个人博客,暂且以次充好吧。哈
  2. 酷壳,里面有好多好文章、资料,自己慢慢发掘吧;
  3. 阮一峰博客,不是专门的挨踢技术博客,但是里面技术类的文章都特别浅显易懂,而且深入浅出,可以学习阮老师的文笔!
  4. Linux大鹏,细致地讲解了不少Linux指令。
  5. 肉饼范凯,ItEye的创始人,技术也很刚刚的!
  6. 池建强
  7. 汤姆大叔,里面有关于JavaScript的一个系列文章,非常不错!
  8. 老赵
  9. ImportNew
  10. 夏の航海士
  11. 冯大辉
  12. 刘未鹏
  13. 淘宝核心系统团队博客
  14. 阿里巴巴集团数据库技术团队
  15. 淘宝JAVA中间件
  16. 淘宝UED团队,博客页面不错!
  17. Matrix67,如果我告诉你这个博主是中文系的,估计你会自卑得无地自容。
    18 王垠
  18. Life As An Outlier
  19. 伯乐在线
  20. 笑遍世界
  21. 疯狂的菠菜
  22. darkmi

知名论坛

  1. 51CTO,侧重服务器、网络管理;
  2. 博客园,侧重.NET技术,前端技术的文章也不少。
  3. 开源社区
  4. V2EX
  5. CSDN;这个是全国最大的IT社区,但是里面的内容参差不齐,仅仅列出,不推荐。

知名资讯博客

  1. 36氪
  2. 月光博客
  3. 爱范儿
  4. 卢松松

奇技淫巧

  1. 如何用好 Google 搜索引擎?
  2. 墙外“谷妹”,墙里佳人“上”,不翻墙,顺利上Google!
  3. 十大高明的Google搜索技巧
  4. GotGitHub,Github的使用教程
  5. Git分支管理策略

必备武器

  1. Google,再次强调一下,必须学会使用Google!
  2. 印象笔记,注意保存资料!也许你现在看到的资料,到明天链接就失效了。
  3. 有道云笔记,也是笔记类软件,网易推出的。备选。
  4. Github,上面有各种各样的代码,喜欢代码的朋友,喜欢开源的朋友,必须会用!
  5. 开源中国-在线工具,各种工具。
  6. 豆瓣读书,可以在这里看书评,选书。

资料荟萃

  1. 分享一些D瓜哥攒的比较好的Web开发资料
  2. Startup News
  3. Hacker News
  4. 极客头条
  5. 网易公开课,很多开放式课程都可以看!
  6. 电驴,虽然现在的电驴被阉割了!但是,里面还是有好多各种各样的学习资料!想学什么,直接在里面搜索就可以!原以为里面的内容都删除了,后来发现登陆后一些链接还是可以照常下!不幸中的万幸啊!
  7. 零基础学习Python
  8. IT学子成长指导
  9. 鲜果网
  10. 我的Linux书架
  11. 豆瓣网,不仅仅读书,还有影评,音乐等等!
  12. 一些文章和各种资源
  13. 20本最好的Linux免费书籍

问答类网站

  1. 知乎,需要注册,不过里面很多问题回答的很深入透彻!推荐注册。
  2. StackOverflow,这个应该是目前全球最大的挨踢问答网了!估计你遇到的任何挨踢相关的问题,都可以在这里找到解答!
  3. Quora,需要注册,并且注册需要跨栏(因为目前只允许米国IP注册)。
      百度知道也是一个问答类网站,但是对于解答挨踢问题来说,太不专业!另外,里面的回答质量也差强人意。就不推荐了。

生活常识

  1. 智慧的提问
  2. 程序员,你会问问题吗?

原文地址 学习方法论

Posted in Uncategorized