Thursday, May 5, 2016

String optimization in Python

Strings are terribly important in programming. A program without some form of string input, manipulation, and output is a rarity.

Of course this means that speed and sanity surrounding string features is important. One important feature of Python is string immutability. This opens up dozens of features, such as using strings as dictionary keys, but there are some downsides.

Immutable strings means that any string manipulation, such as splitting or appending, is making a copy of that string. This can become a performance problem, especially in a world where zero-copy is one of the favorite general optimization techniques. If you've done enough string mutation, you're probably aware of the following techniques:
But in some cases Python uses the immutability to avoid making copies:
>>> a = 'a' * 1024 * 1024  # a 1 megabyte string
>>> z = '' + a
>>> z is a
True
Here, because adding an empty string does not change the value, z is the same exact string object as a. And it doesn't matter how many times you append an empty string:
>>> z = '' + '' + '' + a
>>> z is a
True
It even works when a is the only item in a list:
>>> z = ''.join([a])
>>> z is a
True
But it falls apart when you put an empty string in the list with a:
>>> z = ''.join(['', a])
>>> z is a
False
And unfortunately even the first example seems to make a copy on PyPy:
>>>> a = 'a' * 1024 * 1024  # a 1 megabyte string again
>>>> z = '' + a
>>>> z is a 
False 
Although something more advanced may be going on under the covers, as is often the case with PyPy.

I'm almost done stringing you along, but as a corollary reminder:
Never rely on "is" checks with ints, floats, and strings. "==" and other value checks are what you need. As a general rule, "is" is for objects, None, and sometimes True/False.

Keep on stringifying!

Mahmoud
http://sedimental.org/
https://github.com/mahmoud
https://twitter.com/mhashemi

121 comments:

  1. I think the cpython behaviour is an optimisation possible because of reference counting. Cpython can tell when adding strings if it's the only reference, and can reuse the memory rather than copying in cases like above. Pypy doesn't use reference counting, so can't do the same trick, aiui

    ReplyDelete
    Replies
    1. While it's true that PyPy does not use CPython-style reference counting, I don't think that implies that PyPy can't have this optimization, per se. PyPy still has immutable strings, as that's a property of the Python language, not the runtime.

      Delete
  2. This could be related to interned strings: you can force any string to be unique and in a global table using the intern() function.

    ReplyDelete
    Replies
    1. You are correct about intern() and that may be the culprit with shorter strings, but you'll notice I created a 1MB string to start with. Very much hope our relatively lightweight CPython doesn't magically intern that :)

      Delete
  3. I should thank you for the undertakings you have made in making this article. I am confiding in a similar best work from you later on as well.. Enterprise SEO Services

    ReplyDelete
  4. currently trending technologies are phyton , azure . learn azure through azure training

    ReplyDelete
  5. Just like the annual SEO returns, SEO income per customer also varies. off page seo

    ReplyDelete
  6. I learned World's Trending Technology from certified experts for free of cost. I got a job in decent Top MNC Company with handsome 14 LPA salary, I have learned the World's Trending Technology from python training in btm layout experts who know advanced concepts which can help to solve any type of Real-time issues in the field of Python. Really worth trying Freelance SEO expert in Bangalore

    ReplyDelete
  7. Thank you so much for this useful article. Visit OGEN Infosystem for Web Designing and SEO Services in Delhi, India.
    SEO Service in Delhi

    ReplyDelete
  8. Appslure is a reputed company based in India which provide mobile app development company in mumbai. Our website's layout will be very attractive and responsive, which will gain more visitors and you can get high lead and business from your website. Wonderful post, This article have helped greatly continue writing ..
    Mobile app development company in mumbai

    ReplyDelete
  9. thank you for sharing this blog, it is very useful information for python learning.
    python course bangalore

    ReplyDelete
  10. Excellent blog information shared was very informative and valuable looking forward for next blog thank you.
    Data Analytics Course Online 360DigiTMG

    ReplyDelete
  11. Awesome article with top quality information and I appreciate the writer's choice for choosing this excellent topic found valuable thank you.
    Data Science Training in Hyderabad

    ReplyDelete
  12. I have voiced some of the posts on your website now, and I really like your blogging style. I added it to my list of favorite blogging sites and will be back soon ...

    Business Analytics Course in Bangalore

    ReplyDelete
  13. Happy to chat on your blog, I feel like I can't wait to read more reliable posts and think we all want to thank many blog posts to share with us.

    Data Analytics Course in Bangalore

    ReplyDelete
  14. I'm glad I found this blog! Occasionally, students want to know the keys to writing productive literary essays. Your first-class knowledge of this great job can become a suitable foundation for these people. PMP Training in Hyderabad

    ReplyDelete
  15. I'm glad I found this blog! Occasionally, students want to know the keys to writing productive literary essays. Your first-class knowledge of this great job can become a suitable foundation for these people. Good
    unindent does not match any outer indentation level python

    ReplyDelete
  16. Incredibly in general very intriguing post. I was looking for such an information and took pleasure in scrutinizing this one. Keep posting. An obligation of appreciation is all together for sharing.data analytics course in Hyderabad

    ReplyDelete
  17. wonderful article contains lot of valuable information. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article.
    This article resolved my all queries.good luck an best wishes to the team members.learn digital marketing use these following link
    Digital Marketing Course in Chennai

    ReplyDelete
  18. Thankyou for posting this informative blog, i come to know something new with this. Great Job! Keep it up.

    1000 free youtube subscribers

    ReplyDelete
  19. I have to search sites with relevant information ,This is a
    wonderful blog,These type of blog keeps the users interest in
    the website, i am impressed. thank you.
    Data Science Training in Bangalore

    ReplyDelete
  20. I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page!
    data analytics course in bangalore

    ReplyDelete
  21. I am really enjoying reading your well written articles. It looks like you spend a lot of effort and time on your blog. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work.
    artificial intelligence course in pune

    ReplyDelete
  22. I am glad to discover this page. I have to thank you for the time I spent on this especially great reading !! I really liked each part and also bookmarked you for new information on your site.
    Data Science Training in Chennai

    ReplyDelete
  23. I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page!
    artificial intelligence course in pune

    ReplyDelete
  24. i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
    data scientist course in bangalore

    ReplyDelete
  25. Thanks for posting the best information and the blog is very helpful.data science institutes in hyderabad

    ReplyDelete
  26. Thanks for spending all your pleasant time to make such a Creative content for us. AWS Training in Chennai

    ReplyDelete
  27. I am sure that this is going to help a lot of individuals. Keep up the good work. It is highly convincing and I enjoyed going through the entire blog.

    business analytics course

    ReplyDelete
  28. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
    data science course fees in bangalore

    ReplyDelete
  29. i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
    data scientist course in bangalore

    ReplyDelete
  30. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
    data analytics courses in bangalore

    ReplyDelete
  31. i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
    tableau training in bangalore

    ReplyDelete
  32. Thanks for posting the best information and the blog is very important.artificial intelligence course in hyderabad

    ReplyDelete
  33. Amazing blog.Thanks for sharing such excellent information with us. keep sharing...
    machine learning course in aurangabad

    ReplyDelete
  34. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
    data science course fees in bangalore

    ReplyDelete
  35. Wonderful blog. I delighted in perusing your articles. This is genuinely an incredible perused for me. I have bookmarked it and I am anticipating perusing new articles. Keep doing awesome!
    best machine learning course in aurangabad

    ReplyDelete
  36. I will very much appreciate the writer's choice for choosing this excellent article suitable for my topic. Here is a detailed description of the topic of the article that helped me the most.

    Artificial Intelligence Training in Bangalore

    ReplyDelete
  37. Truly incredible blog found to be very impressive due to which the learners who go through it will try to explore themselves with the content to develop the skills to an extreme level. Eventually, thanking the blogger to come up with such phenomenal content. Hope you arrive with similar content in the future as well.

    Machine Learning Course in Bangalore

    ReplyDelete
  38. I am glad to discover this page. I have to thank you for the time I spent on this especially great reading !! I really liked each part and also bookmarked you for new information on your site.
    Data Science Training in Chennai

    ReplyDelete
  39. First You got a great blog .I will be interested in more similar topics. i see you got really very useful topics, i will be always checking your blog thanks.
    best data science institute in hyderabad

    ReplyDelete
  40. Impressive blog to be honest definitely this post will inspire many more upcoming aspirants. Eventually, this makes the participants to experience and innovate themselves through knowledge wise by visiting this kind of a blog. Once again excellent job keep inspiring with your cool stuff.

    data science certification in bangalore

    ReplyDelete
  41. This is probably one of my favorite blogs you have written. I prefer blogs that don't fall into an echo chamber, and my friend, you have written solid stuff with precise information and good readability. AppSquadz also writes blogs in the same niche, and with new ideas, it might give you more information for the next. Also visit: flutter app development

    ReplyDelete
  42. Thank you so much for ding the impressive job here, everyone will surely like your post.
    business analytics course

    ReplyDelete
  43. Fantastic article I ought to say and thanks to the info. Instruction is absolutely a sticky topic. But remains one of the top issues of the time. I love your article and look forward to more.
    Data Science Course in Bangalore

    ReplyDelete

  44. I like what you guys tend to be up too. This kind of clever work and reporting! Keep up the very good works guys I’ve added you guys to our blogroll.

    Try to check my blog: 부산달리기
    (jk)

    ReplyDelete
  45. i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
    artificial intelligence training in chennai

    ReplyDelete
  46. Thank you quite much for discussing this type of helpful informative article. Will certainly stored and reevaluate your Website.
    Data Analytics Course in Bangalore

    ReplyDelete
  47. Terrific post thoroughly enjoyed reading the blog and more over found to be the tremendous one. In fact, educating the participants with it's amazing content. Hope you share the similar content consecutively.

    data science course in varanasi

    ReplyDelete
  48. I would you like to say thank you so much for my heart. Really amazing and impressive post you have the share. Please keep sharing

    DevOps Training in Hyderabad

    ReplyDelete
  49. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors.
    data science training in chennai

    ReplyDelete
  50. Nice to be visiting your blog once more, it has been months for me. Well this article that i've been waiting for therefore long. I want this article to finish my assignment within the faculty, and it has the same topic together with your article. Thanks, nice share.
    data scientist course in hyderabad

    ReplyDelete
  51. Extremely overall quite fascinating post. I was searching for this sort of data and delighted in perusing this one. Continue posting. A debt of gratitude is in order for sharing. data analytics course in delhi

    ReplyDelete
  52. A good blog always contains new and exciting information, and reading it I feel like this blog really has all of these qualities that make it a blog.

    Artificial Intelligence Training in Bangalore

    ReplyDelete
  53. I am a new user of this site, so here I saw several articles and posts published on this site, I am more interested in some of them, will provide more information on these topics in future articles.
    data science course in london

    ReplyDelete
  54. I was actually browsing the internet for certain information, accidentally came across your blog found it to be very impressive. I am elated to go with the information you have provided on this blog, eventually, it helps the readers whoever goes through this blog. Hoping you continue the spirit to inspire the readers and amaze them with your fabulous content.

    Data Science Course in Faridabad

    ReplyDelete
  55. Nice to be seeing your site once again, it's been weeks for me. This article which ive been waited for so long. I need this guide to complete my mission inside the school, and it's same issue together along with your essay. Thanks, pleasant share.
    Data Science training in Bangalore

    ReplyDelete
  56. Thank you quite much for discussing this type of helpful informative article. Will certainly stored and reevaluate your Website.
    Data Analytics Course in Bangalore

    ReplyDelete
  57. Very wonderful informative article. I appreciated looking at your article. Very wonderful reveal. I would like to twit this on my followers. Many thanks! .
    Data Analytics training in Bangalore

    ReplyDelete
  58. I am glad to discover this page. I have to thank you for the time I spent on this especially great reading !! I really liked each part and also bookmarked you for new information on your site.
    Data Science Course in Gurgaon

    ReplyDelete
  59. it is valuable informative. nice blogs. thanks for sharing these information with all of us.Kinemaster Gold

    ReplyDelete
  60. Nice blog, it is valuable informative. thanks for sharing these information with all of us. whatsapp mod

    ReplyDelete
  61. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors
    cyber security course in gurgaon

    ReplyDelete
  62. Wonderful blog. I delighted in perusing your articles. This is genuinely an incredible perused for me. I have bookmarked it and I am anticipating perusing new articles. Keep doing awesome!.data science training in delhi

    ReplyDelete
  63. i am glad to discover this page : i have to thank you for the time i spent on this especially great reading !! i really liked each part and also bookmarked you for new information on your site.
    data analytics course in patna

    ReplyDelete
  64. I’m excited to uncover this page. I need to to thank you for ones time for this particularly fantastic read !! I definitely really liked every part of it and i also have you saved to fav to look at new information in your site. cyber security course in noida

    ReplyDelete
  65. Extremely overall quite fascinating post. I was searching for this sort of data and delighted in perusing this one. Continue posting. A debt of gratitude is in order for sharing.data science course in warangal

    ReplyDelete
  66. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors
    data scientist course in varanasi

    ReplyDelete
  67. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors
    data analytics course in trivandrum

    ReplyDelete
  68. It is late to find this act. At least one should be familiar with the fact that such events exist. I agree with your blog and will come back to inspect it further in the future, so keep your performance going.

    Best Data Analytics Courses in Bangalore

    ReplyDelete
  69. I am a new user of this site, so here I saw several articles and posts published on this site, I am more interested in some of them, will provide more information on these topics in future articles.
    data science course in london

    ReplyDelete
  70. I want to leave a little comment to support and wish you the best of luck.we wish you the best of luck in all your blogging enedevors
    data analytics course in thiruvananthapuram

    ReplyDelete
  71. This post is so interactive and informative.keep update more information...
    AWS Training in Anna Nagar
    AWS Training in Chennai

    ReplyDelete
  72. Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing.data science course in bhubaneswar

    ReplyDelete
  73. Very informative message! There is so much information here that can help any business start a successful social media campaign!
    data science training in london

    ReplyDelete
  74. I am overwhelmed by your post with such a beautiful subject. I usually visit their blogs and update myself through the information they contain, but today's blog would be most appreciated. Well done!

    Business Analytics Course in Durgapur

    ReplyDelete
  75. I truly like you're composing style, incredible data, thankyou for posting.
    cyber security training malaysia

    ReplyDelete
  76. This post is so interactive and informative.keep update more information...
    hadoop training in tambaram
    Big data training in chennai

    ReplyDelete
  77. Its content is brilliant in many ways. I think it is an attractive and revealing material. Thank you very much for caring about your content and your readers.

    Business Analytics Course in Patna

    ReplyDelete
  78. Great post. keep sharing such a worthy information.
    Mobile app development courses

    ReplyDelete
  79. This is the first time I visit here. I found such a large number of engaging stuff in your blog, particularly its conversation. From the huge amounts of remarks on your articles, I surmise I am by all accounts not the only one having all the recreation here! Keep doing awesome. I have been important to compose something like this on my site and you have given me a thought.

    ReplyDelete
  80. 360DigiTMG, the top-rated organisation among the most prestigious industries around the world, is an educational destination for those looking to pursue their dreams around the globe. The company is changing careers of many people through constant improvement, 360DigiTMG provides an outstanding learning experience and distinguishes itself from the pack. 360DigiTMG is a prominent global presence by offering world-class training. Its main office is in India and subsidiaries across Malaysia, USA, East Asia, Australia, Uk, Netherlands, and the Middle East.

    ReplyDelete
  81. You really make it look so natural with your exhibition however I see this issue as really something which I figure I could never understand. It appears to be excessively entangled and incredibly expansive for me.https://360digitmg.com/course/data-analytics-using-python-r

    ReplyDelete
  82. Great tips and very easy to understand. This will definitely be very useful for me when I get a chance to start my blog.
    full stack developer course

    ReplyDelete
  83. Autodesk Revit Crack stretches a full key to the whole construction scheme facet and provision triggers. The designers, constructing corporations to clarify, condition more¬ well-versed selections beforehand. They give techniques a bit more skillfully. It's novel variations of user¬ demanded exploits, positive Global Boundaries https://freeprosoftz.com/autodesk-revit-crack-key/

    ReplyDelete
  84. Windows 10 Activator Extreme is a tool that locates manufacture and retail keys in many servers for Windows 10 with a broad range of scatterings.
    Windows 10 Activator
    windows 11 activator
    Windows 7 Ultimate Product Key
    windows 8.1 product key

    ReplyDelete
  85. With iTop VPN Crack, clients might conceal their characters and explore namelessly. Utilizing top VPN break, clients might shield their specifically distinguishing data from outer associations. Organizations might impart data to different organizations, restricting their ability to apply fundamental attributes.Itop vpn crack download pc

    ReplyDelete
  86. Boost your professional reputation with a surefire way to pick up some impressive new skills in data science by registering for the Data science courses near me. Learn to collect, clean, and analyze data with tools like Hadoop and Spark. Learn to develop algorithms and build models in machine learning to optimize product performance and gross profit for your organization. Become an expert in techniques like Data Mining, Data Cleansing, and Data Exploring that help refine data, making it possible to present it in an understandable format.

    Best Data Science Training institute in Bangalore

    ReplyDelete
  87. This article will present a closer look at data science courses that give you a comprehensive look at the field. So let's get started.
    data science course in borivali

    ReplyDelete
  88. Fine page, in which did u come happening a distant memory the assessment concerning this posting?i have right of access the majority of the articles with respect to your web website now, and I as a matter of fact in addition to your style. much thanks to you a million and absorb save happening the vivacious deed. Reloader Ultima Versioner

    ReplyDelete
  89. Hello, Dear thank you for your support and help it was helpful for me, to keep posting such informative stuff with us. Also, check our captionme.net for daily social life help.

    ReplyDelete